31:[{"id":"c723ffab194a87c6401b3628b6c3a4ad98ea07344977da83053a097f9e111899e5853006376ce0380155591e80dca132330f2d07b47d8f6a2eebe757773b7443","href":"https://static.observableusercontent.com/files/c723ffab194a87c6401b3628b6c3a4ad98ea07344977da83053a097f9e111899e5853006376ce0380155591e80dca132330f2d07b47d8f6a2eebe757773b7443","name":"History_of_India_Volume_1.txt","type":"text/plain","lastModified":1626161019974.116,"size":501145},{"id":"b420c05591b90e725efe9d3afc163f55806c7a4c843212d03d4b54e0a9801a1e210275f6c024e84a6786a29d3c900b0b6e7f5e66325f2c9845fd5af209d8a67f","href":"https://static.observableusercontent.com/files/b420c05591b90e725efe9d3afc163f55806c7a4c843212d03d4b54e0a9801a1e210275f6c024e84a6786a29d3c900b0b6e7f5e66325f2c9845fd5af209d8a67f","name":"image.png","type":"image/png","lastModified":1626673593056.636,"size":66094}] 2f:{"title":"How to measure winkNLP's speed on browsers?","cells":[{"id":0,"value":"# How to measure winkNLP's speed on browsers?","pinned":false,"hidden":false,"mode":"md"},{"id":16,"value":"md`[WinkNLP](https://github.com/winkjs/wink-nlp) processes raw text at speeds over [525,000 tokens/second](https://github.com/winkjs/wink-nlp#speed--accuracy) for the entire [NLP pipeline](https://winkjs.org/wink-nlp/processing-pipeline.html). The processing includes tokenization, sentence boundary detection, negation handling, sentiment analysis, part-of-speech tagging, and named entity extraction; but *it excludes custom entity recognition*. It offers comparable performance on browsers, which depends on the browser, its version, the host operating system and the processor.\n\nGiven below is a small test jig to measure the performance. **It takes a couple of seconds for the final result to show up.**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":227,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":172,"value":"md`## Processing Pipeline Performance`","pinned":false,"hidden":false,"mode":"ojs"},{"id":230,"value":"FileAttachment(\"image.png\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":19,"value":"{\n let doc;\n let start=Date.now()\n for ( let k = 0; k < 10; k += 1 ) doc = nlp.readDoc( text + k );\n let end=Date.now()\n const perf = Math.round(doc.tokens().length()*12000/(end-start)).toString().replace(/\\B(?=(\\d{3})+(?!\\d))/g, \",\");\n return html`

${perf}^{tokens/second}

_{* Excludes custom entity recognition (cer).`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":220,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":13,"value":"md`### Setup`","pinned":false,"hidden":false,"mode":"ojs"},{"id":4,"value":"winkNLP = (await import('https://cdn.skypack.dev/wink-nlp')).default;","pinned":false,"hidden":false,"mode":"ojs"},{"id":5,"value":"model = (await import('https://cdn.skypack.dev/wink-eng-lite-web-model')).default;","pinned":false,"hidden":false,"mode":"ojs"},{"id":7,"value":"nlp = winkNLP( model );","pinned":false,"hidden":false,"mode":"ojs"},{"id":239,"value":"md`Load \"History of India - Volume I\" book that contains over **125,000 tokens**. This is processed over and over again in a loop of **10 iterations** to compute the speed.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":47,"value":"text = FileAttachment(\"History_of_India_Volume_1.txt\").text()","pinned":false,"hidden":false,"mode":"ojs"},{"id":223,"value":"md`---\n### About\n[**WinkNLP**](https://winkjs.org/) is a **developer friendly** JavaScript library for Natural Language Processing (NLP). Designed specifically to make development of NLP solutions **easier** and **faster**, winkNLP is optimized for the right balance of performance and accuracy. The package can handle large amount of raw text at speeds **over 525,000 tokens/second** for the entire NLP pipeline. And with a **test coverage of ~100%**, [wink-nlp](https://winkjs.org/) is a tool for building production grade systems with confidence.`","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}}