31:[] 2f:{"title":"Parsing a list of wikipedia articles","cells":[{"id":0,"value":"md`# Parsing a list of wikipedia articles\nhere, we demonstrate fetching and parsing data from a number of wikipedia articles in a list.\n\nWe'll take the ***list of apollo astronauts*** we generated [from this example](https://observablehq.com/@spencermountain/parsing-wikipedia-tables?collection=@spencermountain/wtf_wikipedia), and fetch them in sequence, and grab ***their dates of birth*** using [wtf_wikipedia](https://github.com/spencermountain/wtf_wikipedia/).\n\nwe'll also attempt to demonstrate a courteous use of the wikipedia API, to both combine and throttle our requests.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"astronauts = [\n 'Neil Armstrong',\n 'Buzz Aldrin',\n 'Pete Conrad',\n 'Alan Bean',\n 'Alan Shepard',\n 'Edgar Mitchell',\n 'David Scott',\n 'James Irwin',\n 'John Young (astronaut)',\n 'Charles Duke',\n 'Eugene Cernan',\n 'Harrison Schmitt',\n 'Frank Borman',\n 'Jim Lovell',\n 'Bill Anders',\n 'Thomas P. Stafford',\n 'Michael Collins (astronaut)',\n 'Richard F. Gordon Jr.',\n 'Jack Swigert',\n 'Fred Haise',\n 'Stu Roosa',\n 'Al Worden',\n 'Ken Mattingly',\n 'Ron Evans'\n]","pinned":true,"hidden":false,"mode":"ojs"},{"id":29,"value":"md`first, we'll split our list into groups of 5: \n*(use [lodash.chunk](https://lodash.com/docs/4.17.15#chunk), or [this function](http://www.frontcoded.com/splitting-javascript-array-into-chunks.html))*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":32,"value":"groups = chunkBy(astronauts, 5)","pinned":true,"hidden":false,"mode":"ojs"},{"id":38,"value":"md`next, we'll fire-off these requests only 3-at-a-time, using [slow](https://github.com/spencermountain/slow):`","pinned":false,"hidden":false,"mode":"ojs"},{"id":41,"value":"docs = {\n let opts = {'Api-User-Agent': 'wtf_wikipedia example'}\n \n const doit = function(group){\n return wtf.fetch(group, opts) //returns a promise\n }\n //only allow three requests at a time \n return slow.three(groups, doit).then(responses=>{\n //flatten the results\n return [].concat.apply([], responses);\n })\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":52,"value":"md`i know that was a little 'async-foo', but in the end, we have an array of parsed wikipedia articles:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":59,"value":"birthDays = {\n return docs.map(doc => {\n let birthday = null\n // no infobox for Richard Gordon\n if (doc.infobox() && doc.infobox().get('birth_date')) {\n birthday = doc\n .infobox()\n .get('birth_date')\n .text()\n }\n return {\n name: doc.title(),\n birthday: birthday\n }\n })\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":67,"value":"md`finally, we can pretty-print them to a html table:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":70,"value":"html`\n ${birthDays.map((o)=> '').join('\\n')}\n

'+o.birthday+'

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":"wtf=require('wtf_wikipedia')","pinned":false,"hidden":false,"mode":"ojs"},{"id":42,"value":"slow=require('slow')","pinned":false,"hidden":false,"mode":"ojs"},{"id":23,"value":"chunkBy = function(arr, chunkSize) {\n var groups = [], i;\n for (i = 0; i < arr.length; i += chunkSize) {\n groups.push(arr.slice(i, i + chunkSize));\n }\n return groups;\n}\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":7,"value":"wtf.version","pinned":false,"hidden":false,"mode":"ojs"},{"id":10,"value":"style = html``","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}