31:T4a8,Since the start of the year, the online word game [Wordle](https://www.nytimes.com/games/wordle/index.html) has overtaken “crossword” (by 10x), “olympics” (2x) and even “covid” (1.5x) in [Google Trends data](https://trends.google.com/trends/explore?date=2022-01-01%202022-02-13&geo=US&q=wordle,crossword,covid,olympics). [News outlets](https://www.newsweek.com/wordle-255-march-1-guide-answer-clues-hints-tips-starting-word-day-1683538) cover the game down to each day’s answer, and it apparently [helped end a hostage situation](https://www.washingtonpost.com/nation/2022/02/10/wordle-80-year-old-illinois-woman-kidnapped/) in Illinois. The New York Times recently [purchased](https://www.nytimes.com/2022/01/31/business/media/new-york-times-wordle.html) Wordle for a “low seven-figures” sum. First created in June 2021 and made public in October 2021, it’s addicting by nature: one game a day with a good difficulty curve — usually winnable in six tries, but harder to guess in fewer. But it didn’t take off until it added the “Share” button in mid-December, and arrays of cryptic emojis started appearing on people’s feeds. ${tweet("1482377913165099014")} 32:T7d7,Plot.plot({ marks: [ Plot.lineY(numCounts, { x: (d) => new Date(d[0]), y: (d) => d[1], strokeWidth: 3, stroke: "green" }), width > 880 ? Plot.text(annotations, { x: (d) => new Date(numCounts.find((t) => t[0].startsWith(d.textDate))[0]), y: "textCount", lineHeight: 1.3, width: 20, text: (d) => new Date(d.date).toLocaleDateString("en-US", { timeZone: "UTC" }) + " • " + numCounts.find((t) => t[0].startsWith(d.date))[1].toLocaleString() + " tweets\n" + d.text, textAnchor: "start" }) : [ Plot.text(annotations, { x: (d) => new Date(numCounts.find((t) => t[0].startsWith(d.endDate))[0]), y: "textCount", lineHeight: 1.3, width: 20, text: (d, i) => i + 1, textAnchor: "center" }), Plot.dot(annotations, { x: (d) => new Date(numCounts.find((t) => t[0].startsWith(d.endDate))[0]), y: "textCount", r: 10 }) ], Plot.arrow(annotations, { x2: (d) => new Date(numCounts.find((t) => t[0].startsWith(d.date))[0]), y2: (d) => numCounts.find((t) => t[0].startsWith(d.date))[1], x1: (d) => new Date(numCounts.find((t) => t[0].startsWith(d.endDate))[0]), y1: "endCount", // dx: 14, bend: -20, inset: 5, head: null, strokeOpacity: 0.5 }) ], marginLeft: 50, width: width, height: 500, y: { label: "↑ Wordle Tweets per Day", grid: true, nice: true, ticks: 10 }, caption: width > 880 ? "" : html`

${annotations .map( (d, i) => `(${i + 1}) ${new Date(d.date).toLocaleDateString("en-US", { timeZone: "UTC" })} – ${d.text} ` ) .join("
")}

` })33:T45f,Over the last two years, we’ve all had to become uncomfortably familiar with exponential growth, and the corresponding visualizations – so this chart should ring some bells. Players post their scores, triggering others to learn, play, and tweet as well. What started with a few dozen people quickly turned into over a million unique accounts tweeting emoji grids. But that’s just Twitter, and even then only the people willing to tweet. *How many Wordle games have ever been played?* Josh Wardle, the creator of Wordle, hasn’t released any play data, but a New York Times [feature on him and the game](https://www.nytimes.com/2022/01/03/technology/wordle-word-game-creator.html) mentioned the following: > *On Nov. 1, 90 people played.* > > *On Sunday, just over two months later, more than 300,000 people played.* This article came out on January 3rd, meaning the previous Sunday was January 2nd. There were ${f.ls(stats.referenceDayTweets)} Wordle tweets that Sunday, so a conservative estimate for the Play-to-Tweet ratio would be `300000/${stats.referenceDayTweets},` or *${f.ls(stats.playPerTweet)}*.34:T40e,Plot.plot({ marks: [ Plot.barX( d3.rollups( wordleNumSummary, (d) => d3.sum(d, (t) => t.count), (d) => d.rounds ), { y: (d) => d[0], x: (d) => d[1], fill: (d) => d[0], stroke: "black" } ), Plot.text( d3.rollups( wordleNumSummary, (d) => d3.sum(d, (t) => t.count), (d) => d.rounds ), { y: (d) => d[0], x: (d) => d[1], text: (d) => Intl.NumberFormat(undefined, { notation: "compact", compactDisplay: "short" }).format(d[1]), dx: 5, textAnchor: "start" // fill: (d) => d[0] // stroke: "black" } ) ], marginRight: 30, y: { label: "Guesses Taken", paddingInner: 0.2 }, x: { label: "Number of Tweets", nice: true }, color: { transform: (d) => (d == "X" ? 7 : +d), reverse: "true", scheme: "RdYlGn", type: "ordinal" }, width: Math.min(700, width) })35:T788,{ const plot = makeRoundBarPlot(); const plotSelection = d3.select(plot); const topAxis = plotSelection.selectChildren("g").filter((_, i) => i == 1); topAxis.selectAll(".tick").attr("font-size", 16); const yScale = plot.scale("y"); const bandwidth = yScale.apply(yScale.domain[1]) - yScale.apply(yScale.domain[0]); const leftAxis = plotSelection.selectChildren("g").filter((_, i) => i == 0); // Add Dates along y axis leftAxis .append("g") .selectAll("text") .data(yScale.domain) .join("text") .attr("x", -100) .attr("y", (d) => yScale.apply(d) + bandwidth / 2 - 1) .attr("alignment-baseline", "middle") .text( (d) => `${wordleNumToMainDate(d).toLocaleString(undefined, { year: "2-digit", day: "numeric", month: "numeric" })}` ); // Add mean score along y axis leftAxis .append("g") .selectAll("text") .data(yScale.domain) .join("text") .attr("x", -12) .attr("y", (d) => yScale.apply(d) + bandwidth / 2 - 1) .attr("alignment-baseline", "middle") .text( (d, i) => `${d3.reverse(sortedMeanScoreByNum)[i][1].toLocaleString(undefined, { maximumFractionDigits: 2, minimumFractionDigits: 2 })}` ); leftAxis .attr("font-size", 12) .selectAll("text") .attr("fill", "black") .attr("text-anchor", "end"); const searchIdx = d3 .reverse(sortedMeanScoreByNum) .findIndex((d) => getWord(d[0]) == wordSearch.toUpperCase()); leftAxis .selectAll("text") .attr("fill", (d, i) => i % sortedMeanScoreByNum.length == searchIdx ? "red" : "black" ) .style("font-weight", (d, i) => i % sortedMeanScoreByNum.length == searchIdx ? "bold" : "auto" ); const div = d3.select( html`

${plotSelection.node()}

` ); return div.node(); }36:T7c7,The full analysis can be seen in [Appendix B](#appendixB), but the outcome matched the eye test: **uncommon letters** and **double letters** were the two most predictive properties. A regression using both of these properties explains **36%** of the variation in Wordle scores. The other three factors are all slightly predictive. With all five, **45%** of variation can be explained. But after this, things get a little weird. ### Wordle is getting harder? There’s one other factor I have left out so far: *the date order of the words*. When you add this to the model, **47%** of variation can be explained. In fact, date order is the fourth strongest predictor, after uncommon letters, double letters, and word prevalence. And this correlation is positive, meaning average Wordle scores are getting worse over time. **Do I actually think Wordle is getting harder? No.** The words are randomly ordered, and the New York Times did not meaningfully change them. Also, I doubt the creator would have known ahead of time which words were harder than others to this precision. So what is happening? I have two theories: > **THEORY 1**: As more people find and play Wordle, the average player is getting worse at the game. One reason for this could be that the first people to find, play, and share Wordle were more avid puzzle and word-game players, and therefore did better than those that found it later. > **THEORY 2**: The social stigma associated with bad Wordle scores is decreasing. This could be a bit of a feedback loop – as more people claim that Wordle is getting harder, players who might have done poorly will have less hesitation in posting their results. > > The recent talk about the New York Times “making Wordle harder” might have accelerated this trend, as that created an excuse for poor results. When you look at a moving average of average wordle guesses, there seems to be a noticable increase in the week or two following the switch to the New York Times:37:T737,Plot.plot({ marks: [ Plot.ruleX([236], { strokeDasharray: [2, 4], stroke: "steelblue" }), // Plot.lineX(wordleDifficultyAnalysisData.slice(0, 236 - 179), { // x: "num", // y: d3.mean( // wordleDifficultyAnalysisData.slice(0, 236 - 179), // (d) => d.meanScore // ), // stroke: "goldenrod", // strokeDasharray: [3, 5], // strokeWidth: 3 // }), Plot.line(wordleDifficultyAnalysisData, { x: "num", y: d3.mean(wordleDifficultyAnalysisData, (d) => d.meanScore), stroke: "goldenrod", strokeDasharray: [3, 5], strokeWidth: 2 }), // Plot.line(wordleDifficultyAnalysisData, { // x: "num", // y: "meanScoreLOESS", // stroke: "red", // strokeDasharray: [3, 5], // strokeWidth: 3 // }), Plot.line( wordleDifficultyAnalysisData, Plot.windowY( { reduce: "mean", k: 7, anchor: "middle" }, { x: "num", y: "meanScore", stroke: "red", strokeDasharray: [3, 5], strokeWidth: 3 } ) ), Plot.text(wordleDifficultyAnalysisData, { x: "num", y: "meanScore", text: "word", fontSize: 9 }), Plot.text( [ { x: 236, y: 5.1, text: "NYT\nSWITCH\nDATE", fill: "steelblue" }, { x: 193, y: 4.2, text: "AVERAGE SCORE", fill: "goldenrod" } ], { x: "x", y: "y", text: "text", textAnchor: "end", fill: "fill", fontSize: 12, dx: -5 } ) ], y: { grid: true, label: "← Better Average Guesses Taken Worse →", nice: true, labelAnchor: "center" }, x: { transform: (d) => wordleNumToMainDate(d) }, width: Math.max(Math.min(width, 800), 500), insetLeft: 20 })38:T490,{ const data = volumeAnalysisData; const reg = linReg( data, (d) => d.logLoessResidual, (d) => d.meanScore ); return Plot.plot({ marks: [ Plot.text(data, { x: "logLoessResidual", y: "meanScore", text: "word" }), Plot.line(reg, { x: (d) => d[0], y: (d) => d[1], stroke: "red", strokeDasharray: [3, 3] }) ], width, insetLeft: 15, insetRight: 10, x: { grid: true, labelAnchor: "center", label: "← Fewer Tweets Than Expected From Trendline More Tweets Than Expected From Trendline →" }, y: { grid: true, label: "← Better Average Guesses Taken Worse →", nice: true, labelAnchor: "center" }, caption: `Regression Statistics: a: ${reg.a.toLocaleString()}, b: ${reg.b.toLocaleString()}; r^2: ${reg.rSquared.toLocaleString()}; p: ${reg.p2.toLocaleString()} (${ reg.p2 > 0.05 ? "ns" : reg.p2 > 0.01 ? "*" : reg.p2 > 0.001 ? "**" : "***" })` }); }39:T532,Plot.plot({ marks: [ Plot.barY( wordleNumSummary.filter((d) => breakdownKey == "language" && removeEn == true && d.language == "en" ? false : true ), Plot.groupX( { y: "sum", title: (d) => `${d[0][0]} - ${d3.sum(d, (t) => t[1]).toLocaleString()}` }, { x: "wordle_num", y: "count", fill: breakdownKey, offset: "expand", title: (d) => [d[breakdownKey], d.count] } ) ) ], marginLeft: 50, marginRight: 50, marginBottom: 70, width, height: 500, color: { transform: (d) => breakdownKey == "rounds" ? d == "X" ? 7 : d : breakdownKey == "theme" ? d == "l" ? "light" : d == "d" ? "dark" : "unknown" : d, type: "categorical", ...(breakdownKey == "rounds" ? { type: "linear", reverse: "true", scheme: "RdYlGn" } : breakdownKey == "surface" ? { type: "categorical", transform: (d) => surfaceCodes[d] } : {}), legend: true }, x: { tickRotate: 40, tickFormat: (d) => getWord(d) + " " + wordleNumToMainDate(d).toLocaleString("en-US", { dateStyle: "short" }) // tickPadding: 10 } })3a:T564,Plot.plot({ marks: [ Plot.barX( regressionCombinations, Plot.selectMaxX({ y: (d) => d.factorIndex.map((t) => t + 1).join("\n"), x: (d) => d.model.adjust_R2, z: (d) => d.factorNames.length, fill: "steelblue" }) ), Plot.text( regressionCombinations, Plot.selectMaxX({ y: (d) => d.factorIndex.map((t) => t + 1).join("\n"), x: (d) => d.model.adjust_R2, dx: 5, z: (d) => d.factorNames.length, text: (d) => d3.format(".1%")(d.model.adjust_R2), textAnchor: "start" }) ) ], marginLeft: 153, marginRight: 50, y: { label: "Properties", domain: d3 .sort( d3.rollups( regressionCombinations, (d) => d3.greatest(d, (t) => t.model.adjust_R2), (d) => d.factorIndex.length ) // (d) => d[1].model.adjust_R2 ) .map((d) => d[1].factorIndex.map((t) => t + 1).join("\n")), labelAnchor: "top" }, x: { grid: true, // nice: true, label: "Variation Explained (Adjusted R^2)", tickFormat: "%", domain: [0, d3.max(regressionCombinations, (d) => d.model.adjust_R2)] }, caption: "Variation explained (Adjusted R^2 since multiple variables) for the most explanatory combinations at each variable count. Numbers based on single variable graph above." })3b:T70a,volumeAnalysisData = { const data = d3 .rollups( wordleNumSummary, (d) => d3.sum(d, (d) => d.count), (d) => d.wordle_num ) .map((d) => ({ num: d[0], count: d[1], meanScore: new Map(meanScoreByNum).get(d[0]), date: wordleNumToMainDate(d[0]), weekDay: wordleNumToMainDate(d[0]).toLocaleDateString("en-US", { weekday: "long" }), word: getWord(d[0]) })); const loessCounts = d3 .regressionLoess() .x((d) => d.date) .y((d) => d.count) .bandwidth(bandwidth)(data); const logLoessCounts = d3 .regressionLoess() .x((d) => d.date) .y((d) => Math.log10(d.count)) .bandwidth(bandwidth)(data) .map((d) => [d[0], 10 ** d[1]]); data.forEach((d, i) => { d.loessCount = loessCounts[i][1]; d.logLoessCount = logLoessCounts[i][1]; d.blendLoessCount = (d.loessCounts + d.logLoessCounts) / 2; d.loessResidual = (d.count - d.loessCount) / d.count; d.logLoessResidual = (d.count - d.logLoessCount) / d.count; d.blendLoessResidual = (d.loessResidual + d.logLoessResidual) / 2; }); const dayVolumeAvg = d3.rollup( data, (d) => d3.mean(d.map((t) => t.logLoessResidual)), (d) => d.weekDay ); const dayScoreAvg = d3.rollup( data, (d) => d3.mean(d.map((t) => t.meanScore)), (d) => d.weekDay ); data.forEach((d) => { d.weekDayMeanVolume = dayVolumeAvg.get(d.weekDay); d.weekDayMeanScore = dayScoreAvg.get(d.weekDay); d.logLoessResidualweekDayAdjusted = d.logLoessResidual - d.weekDayMeanVolume; }); const scoreModel = jstat.models.ols( data.map((d) => d.meanScore), data.map((d) => [1, d.logLoessResidual]) ); data.forEach((d, i) => { d.scoreResidual = scoreModel.resid[i]; }); return data; }3c:T76e,{ // single var models, p val vs R^2 Plot.plot({ marks: [ Plot.text(regressionPropList, { y: (d) => olsModel(wordleDifficultyAnalysisData, "meanScore", [d[1]]).t.p[1], x: (d) => olsModel(wordleDifficultyAnalysisData, "meanScore", [d[1]]).R2, text: "0" }) ], inset: 10, marginLeft: 50, x: { label: "R^2", nice: true, grid: true }, y: { label: "p value", grid: true, type: "log", tickFormat: "n" } }); // var combination superset, var count vs adjusted R^2 Plot.plot({ marks: [ Plot.text(regressionCombinations, { y: (d) => d.model.R2, x: (d) => d.factorIndex.length, text: "factorIndex", fontSize: 12, fill: (d) => d.factorIndex.length, filter: (d) => d.factorIndex // order: (d) => olsModel(wordleDifficultyAnalysisData, "meanScore", [d[1]]).R2 }) ], inset: 10, marginLeft: 50, height: 800, width, x: { label: "Variable Count", nice: true, grid: true }, y: { label: "Adjusted R^2", nice: true, grid: true } }); // single var models, R^2 vs R^2 lost when removed from full model Plot.plot({ marks: [ Plot.text( regressionCombinations.filter((d) => d.factorIndex.length == 1), { y: (d) => d.model.adjust_R2, x: (d) => fullFactorTimeAwareModel.adjust_R2 - d.inverseModel.adjust_R2, text: "factorNames", fontSize: 12 } ) ], inset: 10, marginLeft: 50, height: 800, width, x: { label: "Adjusted R^2 Lost When Removed From Full Model", nice: true, grid: true }, y: { label: "Adjusted R^2", nice: true, grid: true } }); return "extra difficulty viz"; }3d:T419,wordleDifficultyAnalysisData = meanScoreByNum.map((d, i) => ({ num: d[0], meanScore: d[1], date: wordleNumToMainDate(d[0]), meanScoreLOESS: wordleDifficultyOverTimeLOESS[i][1], // meanConsistentPlayerScore: new Map(meanConsistentPlayerScoreByNum).get(d[0]), // meanScoreAllMinusConsistent: // d[1] - new Map(meanConsistentPlayerScoreByNum).get(d[0]), word: getWord(d[0]), harmonicMeanLetterFrequency: getHarmonicMeanLetterFreq(getWord(d[0])), vowelStructure: getVowelStructure(getWord(d[0])), vowelStructureFrequency: getVowelStructureFrequency(getWord(d[0])), uniqueLetterCount: getUniqueLetters(getWord(d[0])), wordLogFrequency: getWordLogFrequency(getWord(d[0])), wordPrevalence: getWordPrevalence(getWord(d[0])), adjacentWordCount: getAdjacentWordCount(getWord(d[0])), adjacent2WordCount: getAdjacentWordCount(getWord(d[0]), 2), adjacentAnswerWordCount: getAdjacentWordCount(getWord(d[0]), 1, "ANS"), adjacentFilteredWordCount: getAdjacentWordCount( getWord(d[0]), 1, "ALL", filterFn ) }))3e:T510,makeRoundBarPlot = () => Plot.plot({ marks: [ Plot.barX( wordleNumSummary, Plot.groupY( { x: "sum", title: (d) => `${d[0][0]} (${d3.sum(d, (t) => t[1]).toLocaleString()})` }, { y: "wordle_num", x: "count", fill: "rounds", offset: "expand", title: (d) => [d["rounds"], d.count], stroke: "black", strokeWidth: 0.7 // strokeOpacity: 0.5 } ) ), Plot.ruleX([0.25, 0.5, 0.75], { stroke: "gray" }) ], marginLeft: 155, // marginTop: 100, width: width + 100, color: { transform: (d) => (d == "X" ? 7 : +d), reverse: "true", scheme: "RdYlGn", type: "ordinal" }, x: { axis: "top", ticks: topLabelPositions, tickFormat: (d, i) => (i < 6 ? i + 1 : "X"), label: null, tickSize: 0, tickPadding: 3 }, y: { tickFormat: (d) => `${getWord(d)}`, label: null, domain: sortedMeanScoreByNum.map((d) => d[0]), reverse: true, tickSize: 0, tickPadding: 45, paddingInner: 0.15 }, style: { // background: "#383133", // color: "white" } })3f:T412,getAdjacentWords = (word, steps = 1, corpus = "ALL") => { const adjWords = []; word = word.toUpperCase(); if (steps > 2 || steps < 1) { throw "let's not"; } const wordSet = new Set(corpus == "ALL" ? words.allowed : words.answers); const permuteWord = (w) => d3 .range(5) .flatMap((idx) => alphabet.map((letter) => { // this letter has already been changed if (w[idx] == w[idx].toLowerCase()) return; // this letter would be the same as before if (w[idx] == letter.letter) return; const letters = [...w]; letters[idx] = letter.letter.toLowerCase(); return letters.join(""); }) ) .filter((w) => w); let i = 0; let permutations = [word]; while (i < steps) { permutations = permutations.flatMap((w) => permuteWord(w)); permutations.forEach((w) => { w = w.toUpperCase(); if (wordSet.has(w)) { adjWords.push(w); } }); i += 1; } return [...new Set(adjWords)]; }40:T102c,{ ("CONSISTENCY GRAPH 1"); // Plot.plot({ // marks: [ // Plot.barY( // dataToggle ? consistencyData2 : wordleNumSummary, // Plot.groupX( // { // y: "sum", // title: (d) => // `${d[0][0]} - ${d3.sum(d, (t) => t[1]).toLocaleString()}` // }, // { // x: "wordle_num", // y: "count", // fill: "rounds", // // offset: "expand", // title: (d) => [d.rounds, d.count] // } // ) // ) // ], // marginLeft: 50, // marginBottom: 70, // width, // height: 500, // y: { // // type: "symlog", // // ticks: [1, 10, 100, 1000, 10000], // grid: true // }, // color: { // transform: (d) => (d == "X" ? 7 : d), // type: "linear", // reverse: "true", // scheme: "RdYlGn", // legend: true // }, // x: { // tickRotate: 40, // tickFormat: (d) => // getWord(d) + // " " + // wordleNumToMainDate(d).toLocaleString("en-US", { dateStyle: "short" }) // // tickPadding: 10 // } // }); ("CONSISTENCY GRAPH 2"); // Plot.plot({ // marks: [ // Plot.barX( // // d3.rollups( // // consistencyData2, // // (d) => d3.sum(d, (t) => t.count), // // (d) => d.rounds // // ), // // { fill: (d) => d[0], x: (d) => d[1] } // [ // ...wordleNumSummary.map((d) => ((d.type = "All Players"), d)), // ...consistencyData2.map((d) => ((d.type = "Consistent Players"), d)) // ], // Plot.groupY( // { // x: "sum" // }, // { // x: "count", // y: "type", // fill: "rounds", // offset: "expand", // title: "rounds" // // order: (d) => (d.rounds == "X" ? 1 : +d) // } // ) // ) // ], // marginLeft: 100, // y: { // label: null // }, // color: { // // domain: [0, 7], // transform: (d) => (d == "X" ? 7 : +d), // type: "linear", // reverse: "true", // scheme: "RdYlGn", // legend: true, // ticks: 7, // tickFormat: (d) => (d == 7 ? "X" : d) // }, // legend: {} // }) ("CONSISTENCY GRAPH 3"); // Plot.plot({ // marks: [ // Plot.barX( // d3.rollups( // dataToggle ? consistencyData2 : wordleNumSummary, // (d) => d3.sum(d, (t) => t.count), // (d) => d.rounds // ), // { // y: (d) => d[0], // x: (d) => d[1], // fill: (d) => d[0], // stroke: "black" // } // ), // Plot.text( // d3.rollups( // dataToggle ? consistencyData2 : wordleNumSummary, // (d) => d3.sum(d, (t) => t.count), // (d) => d.rounds // ), // { // y: (d) => d[0], // x: (d) => d[1], // text: (d) => // Intl.NumberFormat(undefined, { // notation: "compact", // compactDisplay: "short" // }).format(d[1]), // dx: 5, // textAnchor: "start" // // fill: (d) => d[0] // // stroke: "black" // } // ) // ], // marginRight: 30, // y: { // label: "Guesses Taken", // paddingInner: 0.2, // type: "band" // }, // x: { // label: "Number of Tweets" // // nice: true // }, // color: { // transform: (d) => (d == "X" ? 7 : +d), // reverse: "true", // scheme: "RdYlGn", // type: "ordinal" // }, // width: Math.min(700, width) // }) ("CONSISTENCY GRAPH 4"); // Plot.cell(consistencyAggregates, { // x: "wordle_span", // y: "wordle_count", // fill: (d) => Math.log(d.count) // }).plot({ // width: 750, // height: 600, // marginBottom: 35, // y: { reverse: true }, // x: { tickRotate: -90, labelOffset: 32 } // }) return "consistency graphs"; }41:[{"id":"2efadf0d5ddec59c49ae004bacbb4accaca8cc6defc106e0ab9e5bb378823819b514c6e3075e8036a65b7c78c645679461a87b6a92200cc8ed44397ff263c3ad","href":"https://static.observableusercontent.com/files/2efadf0d5ddec59c49ae004bacbb4accaca8cc6defc106e0ab9e5bb378823819b514c6e3075e8036a65b7c78c645679461a87b6a92200cc8ed44397ff263c3ad","name":"fiveLetterPrevalenceData.csv","type":"text/csv","lastModified":1645578496158.076,"size":96763},{"id":"e5d46946650ca3f10c77646f052203b583bf0f24dbb3aca111f481f397665700c2fb8d33452b50b1828534d46143947e5832fc7a1d1f3d67577feb4797abf08b","href":"https://static.observableusercontent.com/files/e5d46946650ca3f10c77646f052203b583bf0f24dbb3aca111f481f397665700c2fb8d33452b50b1828534d46143947e5832fc7a1d1f3d67577feb4797abf08b","name":"wordleNumSummary (4).csv","type":"text/csv","lastModified":1646722662457.664,"size":3686950},{"id":"3ad062dd6601f66e52240505051c3f2b4668e5cdfebd465313fd5b029f7cafb9a42b2c3889911753491d61dd74bc0d074b3654c23bd7b8849e60ba52149019b6","href":"https://static.observableusercontent.com/files/3ad062dd6601f66e52240505051c3f2b4668e5cdfebd465313fd5b029f7cafb9a42b2c3889911753491d61dd74bc0d074b3654c23bd7b8849e60ba52149019b6","name":"matrixArtData.csv","type":"text/csv","lastModified":1645410780997.368,"size":31006},{"id":"802655142d5fa9b84eaf16915397f28b142f46e7eb7b2c7e078a8134ab16014118006e42f9d4ef44c3808313fecdf0c73a6b4e079ae3444b09165344aae6eb23","href":"https://static.observableusercontent.com/files/802655142d5fa9b84eaf16915397f28b142f46e7eb7b2c7e078a8134ab16014118006e42f9d4ef44c3808313fecdf0c73a6b4e079ae3444b09165344aae6eb23","name":"wordleByHour (4).csv","type":"text/csv","lastModified":1646722694735.606,"size":140799}] 2f:{"title":"Wordle, 15 Million Tweets Later","cells":[{"id":0,"value":"# Wordle, 15 Million Tweets Later","pinned":false,"hidden":false,"mode":"md"},{"id":3310,"value":"showSVGMosaic(matrixArtData, 1, 18)","pinned":false,"hidden":false,"mode":"ojs"},{"id":4072,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":9074,"value":"p1 = md`So, to really understand Wordle, I had to look at the tweets – all ${f.nf(\n stats.totalTweets\n)} of them^[[1](#f1)].\n\n---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5003,"value":"## I. Wordle Goes Viral","pinned":false,"hidden":false,"mode":"md"},{"id":5128,"value":"Let’s start with the basics – what has the growth of Wordle looked like?","pinned":false,"hidden":false,"mode":"md"},{"id":5020,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":5428,"value":"$33","pinned":false,"hidden":false,"mode":"md"},{"id":9081,"value":"p2 = md`From the first (formally) shared Wordle game on Dec 15, 2021 until the most recent data on ${f.lds(\n stats.lastDay\n)}, there have been ${f.ls(\n stats.totalTweets\n)} tweets^[[2](#f2)]. Applying our ratio here gives us a rough estimate: **${f.nf(\n stats.totalPlays\n)} Wordle plays.**\n\nThis is a pretty massive sum. If we assume the average Wordle game takes five minutes to play^[[3](#f3)], then over **${f.ls(\n stats.totalYears\n)} years** of Wordle have been played.\n\n---\n\nNext, let’s dive deeper into the data, specifically around the number of guesses needed to solve Wordle.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5688,"value":"md`\n

❗️❗️ Warning! The following analysis displays the Wordle solutions for *previous days*. The current day will never be shown, but if you plan on playing old Wordles on an archive site, beware. ❗️❗️

\n\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5722,"value":"## II. What makes a Wordle hard?\n\nEach Wordle game gives the player six tries to guess the word. If you’ve played a decent amount, you’ve probably developed an intuition for how “good” certain results are. \n\nPrevailing thought on Twitter seems to be that four guesses is a typical outcome. This is the best analogy I’ve found:\n\n${tweet('1486439723581288451')}\n\nWith that in mind, here’s the distribution of Wordle results among all tweets:","pinned":false,"hidden":false,"mode":"md"},{"id":5754,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":5826,"value":"p3 = md`As expected, four is the most common guess count, with frequency decreasing rapidly on either side. Getting a Wordle-in-one is quite rare^[[4](#f4)], with less than a third the volume of losses (denoted with an “X”).\n\nNow, let’s break that down by target word.\n\n*This chart shows the score distribution for each word, along with the date it was played, and the average score^[[5](#f5)]. It is initially sorted from most difficult word to least. Change the sorting value with the options below the chart.*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":10078,"value":"viewof wordSearch = Inputs.text({\n placeholder: \"Search for a past word to highlight\",\n maxlength: 5,\n validate: (d) => d.value.length == 0 || d.value.length == 5,\n datalist: meanScoreByNum.map((d) => getWord(d[0]))\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":5906,"value":"viewof sort = Inputs.radio([\"Difficulty\", \"Date\"], {\n label: \"Sort by\",\n value: \"Difficulty\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":2502,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":5934,"value":"p4 = md`Just from the eyeball test, a few characteristics stick out among the most difficult words – double letters (i.e. FERRY, SWILL) and uncommon letters (i.e. PROXY, CRAZE).\n\nFor a more rigorous analysis, I ran a series of linear regression tests to predict the average score, using the following properties:\n- **Uncommon letters** – Average frequency of the letters in English words\n- **Double letters** – Occurrence of a letter multiple times in the word\n- **Word structure** – Placement of consonants and vowels in the word \n- **Word adjacencies** – How many words are one letter change away from the word? (i.e. BRAIN → BRAID)^[[6](#f6)]\n- **Word Knowledge** – How well known is this word?^[[7](#f7)]`","pinned":false,"hidden":false,"mode":"ojs"},{"id":9993,"value":"$36","pinned":false,"hidden":false,"mode":"md"},{"id":8658,"value":"$37","pinned":false,"hidden":false,"mode":"ojs"},{"id":8713,"value":"---\n\nOne obvious but important caveat to the above analysis, and something I began to touch on, is that *all Wordle tweets are self-reported data*. While this fact seems to have a small but meaningful impact on the difficulty between words, it hugely impacts the overall dataset. Let’s dig into that.","pinned":false,"hidden":false,"mode":"md"},{"id":5660,"value":"## III. Are Wordle Tweets Just the Highlights?\n\nA common saying about sharing in the internet age is that social media is a highlight reel. \nPeople only post their best moments.\n\nDoes this apply to Wordle? Are people less likely to share their mediocre performances or failures?\n\nHere’s one way to approach the question. If people are less likely to share their results when they do poorly, we would expect fewer people to do so on days with harder Wordle answers. Let’s look at that tweet time series again:","pinned":false,"hidden":false,"mode":"md"},{"id":7443,"value":"Plot.plot({\n marks: [\n Plot.lineY(numCounts, {\n x: (d) => new Date(d[0]),\n y: (d) => d[1],\n strokeWidth: 3,\n stroke: \"green\"\n })\n ],\n marginLeft: 50,\n width: width,\n height: 350,\n y: {\n label: \"↑ Wordle Tweets per Day\",\n grid: true,\n nice: true,\n ticks: 10\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":7449,"value":"p5 = md`The first thing you notice is the massive increase in Wordle tweets, followed by a plateau and gradual decline. But the second thing you notice is that there’s a lot of variation between nearby dates. Could this variation be due to difficulty?\n\nTo test this, I fit a curve to the data^[[8](#f8)]:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":7459,"value":"Plot.plot({\n marks: [\n Plot.lineY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.count,\n strokeWidth: 1,\n stroke: \"green\"\n }),\n Plot.lineY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.logLoessCount,\n strokeWidth: 2,\n stroke: \"blue\",\n strokeDasharray: [4, 5]\n })\n ],\n marginLeft: 50,\n width: width,\n height: 350,\n y: {\n label: \"↑ Wordle Tweets per Day\",\n grid: true,\n nice: true,\n ticks: 10,\n domain: [0, 350000]\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":7958,"value":"I then compared the residuals (the difference between actual and expected values) to the average score for each Wordle. Sure enough, Wordle difficulty alone explained **29%** of the variation in tweet volume. More difficult words, like FERRY, ABBEY, and SEIGE saw fewer tweets than expected, while easier words like TAPIR, DRINK, and PEACH saw more.","pinned":false,"hidden":false,"mode":"md"},{"id":7978,"value":"$38","pinned":false,"hidden":false,"mode":"ojs"},{"id":9874,"value":"The strong relationship between Wordle difficulty and tweet volume bolsters the idea of the “highlight reel” effect, that players are less likely to share poor results. ","pinned":false,"hidden":false,"mode":"md"},{"id":8023,"value":"---\n\n## Further Explorations\n\nThere’s lots more analysis that could be done here. I'm particularly interested in analysis of the emoji grids, such as how people arrive at the answer, but it proved too unwieldy to visualize for this piece. I also generated some interactive exploratory charts while making this, which you can find in [Appendix A](#appendixA).\n\nIf you have ideas for areas to explore, or a question or comment more generally, please [reach out](https://twitter.com/RobertLesser_)! Hopefully, you've learned something new, or at least have a new Wordle fact to tell your friends.","pinned":false,"hidden":false,"mode":"md"},{"id":9954,"value":"showSVGMosaic(matrixArtData, 1, 18)","pinned":false,"hidden":false,"mode":"ojs"},{"id":11566,"value":"### Acknowledgements\n\nBig thanks to [Toph Tucker](https://observablehq.com/@tophtucker) for extremely helpful feedback!\n\nAlso thanks to Benn Stancil's [wordle/twitter piece](https://benn.substack.com/p/how-to-play-wordle?utm_source=url&s=r) for giving me additional ideas.\n\n### Further Reading\n\n[David Waldron](https://twitter.com/davidawaldron) wrote an [excellent article](https://waldrn.com/what-makes-a-wordle-word-hard/) exploring a similar set of questions, well worth checking out.\n","pinned":false,"hidden":false,"mode":"md"},{"id":8546,"value":"### Footnotes","pinned":false,"hidden":false,"mode":"md"},{"id":9958,"value":"f1 = footnote(\n \"p1\",\n 1,\n \"Data was collected almost entirely with the Twitter API, which I began using in January. For data that was outside of the retention window when I began, I used a semi-automated method to gather the tweet data. This was more difficult than the API, but luckily there weren’t too many tweets to gather in the early days.\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9966,"value":"f2 = footnote(\n \"p2\",\n 2,\n \"While there were definitely Wordle games played before the Twitter sharing capability was added, the volume was most likely low enough that they have a negligible impact on the total number of plays.\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9974,"value":"f3 = footnote(\n \"p2\",\n 3,\n \"The five minutes per game quantity was determined through a very robust survey of myself, and a half dozen Wordle-playing friends.\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9977,"value":"f4 = footnote('p3', 4, 'If you thing the Wordle-in-one quantity should be rarer still, and his being overshared compared to other results, I agree with you! Read on to section III.')","pinned":false,"hidden":false,"mode":"ojs"},{"id":9986,"value":"f5 = footnote(\n \"p3\",\n 5,\n \"To determine the average score, I treated a loss as 7 guesses.\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9998,"value":"f6 = footnote(\n \"p4\",\n 6,\n \"The idea behind this one is that getting to four letters can be a bit of a trap, if there are many words that fit the known pattern. Two refinements to the base idea that I made were: 1) Only consonant changes were considered, and 2) the adjacent words had to be more frequently found in English text than the target word.\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":10033,"value":"f7 = footnote(\n \"p4\",\n 7,\n html`Word prevalence data came from a paper on Word prevalence norms for 62,000 English lemmas, which I found through Yuri Vishnevsky's notebook.`\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":10051,"value":"f8 = footnote(\n \"p5\",\n 8,\n html`The curve used was a LOESS curve, fit to a log10 transformation of volume data. See Appendix C`\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":7488,"value":"appendixA = md`## Appendix A - Bonus Charts`","pinned":false,"hidden":false,"mode":"ojs"},{"id":8267,"value":"I went through an unholy number of charts in the process of creating this project. Below are a couple interactive, exploratory visualizations I decided to keep around.\n\n*These are not mobile optimized!*","pinned":false,"hidden":false,"mode":"md"},{"id":3599,"value":"### Wordle Breakdown by Number","pinned":false,"hidden":false,"mode":"md"},{"id":2073,"value":"viewof breakdownKey = Inputs.select(\n Object.keys(wordleNumSummary[0]).slice(1, -1),\n {\n label: \"Breakdown\"\n }\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1261,"value":"viewof removeEn = Inputs.toggle({\n label: \"Remove English?\",\n disabled: breakdownKey !== \"language\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":2017,"value":"$39","pinned":false,"hidden":false,"mode":"ojs"},{"id":3595,"value":"### Wordle Count by Hour and Number","pinned":false,"hidden":false,"mode":"md"},{"id":8507,"value":"viewof dateRange = rangeSlider({\n min: d3.min(wordleByHour, (d) => d.wordle_num),\n max: d3.max(wordleByHour, (d) => d.wordle_num),\n value: [179, 200],\n format: (d) =>\n wordleNumToMainDate(d).toLocaleString(\"en-US\", { dateStyle: \"short\" }),\n step: 1,\n title: \"Wordle Date Range\",\n description:\n \"Drag the ends of the slider or the middle of the slider to change the date range.\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1538,"value":"viewof k = Inputs.range([1, 24], {\n step: 1,\n label: \"Hours to average\",\n sss: \"eee\",\n value: 6\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":9910,"value":"Plot.plot({\n marks: [\n Plot.lineY(\n wordleByHour.filter(\n (d) => d.wordle_num > dateRange[0] && d.wordle_num < dateRange[1]\n ),\n Plot.windowY(\n { reduce: \"mean\", k, anchor: \"middle\" },\n {\n x: (d) => new Date(d.hour.replace(\" \", \"T\")),\n y: (d) => d.count,\n stroke: \"wordle_num\",\n sort: \"hour\",\n title: \"wordle_num\"\n }\n )\n ),\n Plot.text(\n wordleByHour.filter(\n (d) => d.wordle_num > dateRange[0] && d.wordle_num < dateRange[1]\n ),\n Plot.selectMaxY(\n Plot.windowY(\n { reduce: \"mean\", k, anchor: \"middle\" },\n {\n x: (d) => new Date(d.hour.replace(\" \", \"T\")),\n y: (d) => d.count,\n z: \"wordle_num\",\n sort: \"hour\",\n title: (d) => getWord(d.wordle_num),\n text: (d) => getWord(d.wordle_num),\n dy: -10\n }\n )\n )\n )\n ],\n width,\n caption: \"Count by hour and wordle word\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":9906,"value":"Notice the unique double-spike for the word GRIPE on Dec 24. It seems to have been caused by a [Guardian article](https://www.theguardian.com/games/2021/dec/23/what-is-wordle-the-new-viral-word-game-delighting-the-internet), as there is a [corresponding spike in searches](https://trends.google.com/trends/explore?date=2021-12-15%202021-12-30&geo=GB&q=wordle) in the UK. I believe this explains why GRIPE had a far higher tweet volume than expected in the regression analysis in Section III.","pinned":false,"hidden":false,"mode":"md"},{"id":8591,"value":"appendixB = md`## Appendix B - Word Difficulty Analysis`","pinned":false,"hidden":false,"mode":"ojs"},{"id":8599,"value":"I had to get further into the statistical weeds than I initially prepared for, but I got to make some nice regression plots, so it was all worth it.","pinned":false,"hidden":false,"mode":"md"},{"id":6400,"value":"### Single Variable Regressions on Difficulty\n\n*Select a property from the dropdown to see how that variable correlates with average score. The red line marks the linear regression, and the significance test statistics are in the caption below.*","pinned":false,"hidden":false,"mode":"md"},{"id":6636,"value":"viewof xProp = Inputs.select(regressionPropList, {\n label: \"Regression Property\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":6584,"value":"linRegPlot(wordleDifficultyAnalysisData, xProp, \"meanScore\", \"word\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":11257,"value":"### Comparing Explanatory Power of Variables","pinned":false,"hidden":false,"mode":"md"},{"id":8625,"value":"Of these individual properties, letter rarity and double letters are by far the most statistically significant. Word prevalence and Wordle date are also significant individually. However, once you account for those, more properties become significant, albeit less-so. ","pinned":false,"hidden":false,"mode":"md"},{"id":10930,"value":"Plot.plot({\n marks: [\n Plot.barX(\n regressionCombinations.filter((d) => d.factorNames.length == 1),\n {\n y: \"factorNames\",\n x: (d) => d.model.R2,\n fill: \"steelblue\"\n }\n ),\n Plot.text(\n regressionCombinations.filter((d) => d.factorNames.length == 1),\n {\n y: \"factorNames\",\n x: (d) => d.model.R2,\n dx: 5,\n text: (d) => d3.format(\".1%\")(d.model.R2),\n textAnchor: \"start\"\n }\n )\n ],\n marginLeft: 153,\n marginRight: 50,\n y: {\n label: \"Property\",\n tickFormat: (d, i) => i + 1 + \" – \" + d,\n domain: regressionCombinations\n .filter((d) => d.factorNames.length == 1)\n\n .map((d) => d.factorNames),\n labelAnchor: \"top\"\n },\n x: {\n grid: true,\n // nice: true,\n label: \"Variation Explained (R^2)\",\n tickFormat: \"%\",\n domain: [0, d3.max(regressionCombinations, (d) => d.model.adjust_R2)]\n },\n caption: \"Variation explained (R^2) for each individual property\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":10877,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs"},{"id":11250,"value":"### Custom Multivariate Linear Regression Model","pinned":false,"hidden":false,"mode":"md"},{"id":11081,"value":"You can play with the model variables here, and then compare the custom model residual to an additional regression property:","pinned":false,"hidden":false,"mode":"md"},{"id":10400,"value":"viewof multiRegressionProps = Inputs.checkbox(regressionPropList, {\n label: \"Model Properties\",\n multiple: true,\n value: [\"wordPrevalence\", \"uniqueLetterCount\"]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":11085,"value":"*Adjusted R² of Custom Model:* `${d3.format('.3')(customDifficultyModel.adjust_R2)}`","pinned":false,"hidden":false,"mode":"md"},{"id":11066,"value":"viewof xProp2 = Inputs.select(regressionPropList, {\n label: \"Regression Property\",\n value: \"harmonicMeanLetterFrequency\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":10728,"value":"linRegPlot(wordleDifficultyResiduals, xProp2, \"customResidual\", \"word\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":10062,"value":"appendixC = md`## Appendix C - Wordle Volume Analysis`","pinned":false,"hidden":false,"mode":"ojs"},{"id":11245,"value":"appendixC2 = md`### LOESS Curve Fitting`","pinned":false,"hidden":false,"mode":"ojs"},{"id":11169,"value":"I fit a LOESS curve to the volume data in order to get a rough average volume at all times. I found a LOESS on the log of the volume both had better performance and made more sense given the large difference in magnitudes over the course of the data.\n\n*The Red dotted line below is the log LOESS, and the blue is the regular LOESS. The better performance of the log LOESS is especially noticeable around the early values on the log scale.*","pinned":false,"hidden":false,"mode":"md"},{"id":7630,"value":"viewof logScale = Inputs.toggle({ label: \"Log Scale\" })","pinned":false,"hidden":false,"mode":"ojs"},{"id":11202,"value":"viewof yMax = Inputs.range([1000, d3.max(volumeAnalysisData, (d) => d.count)], {\n step: 1,\n label: \"Max Y\",\n value: d3.max(volumeAnalysisData, (d) => d.count),\n transform: Math.log\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":4765,"value":"Plot.plot({\n marks: [\n Plot.lineY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.count,\n strokeWidth: 1,\n stroke: \"black\",\n opacity: 0.5\n }),\n Plot.dot(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.count,\n title: (d) => d.word,\n strokeWidth: 1,\n stroke: \"black\",\n opacity: 0.5\n }),\n Plot.lineY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.loessCount,\n strokeWidth: 1,\n stroke: \"blue\",\n strokeDasharray: [3, 2]\n }),\n Plot.lineY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d.logLoessCount,\n strokeWidth: 1,\n stroke: \"red\",\n strokeDasharray: [3, 2]\n })\n ],\n // width: width,\n marginLeft: 50,\n y: {\n grid: true,\n // nice: true,\n type: logScale ? \"log\" : \"linear\",\n domain: [200, yMax],\n label: \"Volume per day\"\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":11229,"value":"The residual graph below shows the Log LOESS is well calibrated over time:","pinned":false,"hidden":false,"mode":"md"},{"id":4828,"value":"Plot.plot({\n marks: [\n [\n \"_REMOVETOSHOW_loessResidual\",\n \"logLoessResidual\",\n \"_REMOVETOSHOW_blendLoessResidual\"\n ].flatMap((r, i) => [\n Plot.dotY(volumeAnalysisData, {\n x: (d) => d.date,\n y: (d) => d[r],\n strokeWidth: 1,\n stroke: [\"blue\", \"red\", \"purple\"][i]\n }),\n Plot.lineY(\n linReg(\n volumeAnalysisData,\n (d) => d.date,\n (d) => d[r]\n ),\n {\n x: (d) => new Date(d[0]),\n y: (d) => d[1],\n strokeWidth: 1,\n stroke: [\"blue\", \"red\", \"purple\"][i],\n strokeDasharray: [4, 3]\n }\n )\n ])\n ],\n // height: 600,\n marginLeft: 50,\n y: {\n label: \"Difference between expected volumes and actual volumes\",\n grid: true,\n nice: true,\n domain: [-1, 1]\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":2891,"value":"## Code","pinned":false,"hidden":false,"mode":"md"},{"id":5137,"value":"annotations = [\n {\n date: \"2021-12-15\",\n endDate: \"2021-12-16\",\n endCount: 55000,\n text: \"Sharing ability is added to Wordle\",\n textDate: \"2021-12-16\",\n textCount: 64000\n },\n {\n date: \"2022-01-03\",\n endDate: \"2022-01-01\",\n endCount: 117000,\n text:\n \"NYT article “Wordle is a Love Story” \\nis published, one of the first major \\narticles about the game\",\n textDate: \"2021-12-24\",\n textCount: 134000\n },\n {\n date: \"2022-01-31\",\n endDate: \"2022-02-02\",\n endCount: 245000,\n text: \"NYT buys Wordle\",\n textDate: \"2022-01-31\",\n textCount: 238000\n }\n]","pinned":false,"hidden":false,"mode":"ojs"},{"id":5566,"value":"stats = {\n const o = {};\n o.totalTweets = d3.sum(numCounts, (d) => d[1]);\n const referenceDayIdx = 18;\n o.referenceDayTweets = numCounts[referenceDayIdx][1];\n o.playPerTweet = 300000 / o.referenceDayTweets;\n o.totalPlays = o.totalTweets * o.playPerTweet;\n o.minPerPlay = 5;\n o.totalMin = o.totalPlays * o.minPerPlay;\n o.totalYears = Math.floor(o.totalMin * (1 / 60) * (1 / 24) * (1 / 365.24));\n o.lastDay = new Date(numCounts.slice(-1)[0][0]);\n return o;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":5615,"value":"f = ({\n ls: (n) => n.toLocaleString(),\n lds: (n) =>\n n.toLocaleDateString(\"en-US\", { dateStyle: \"medium\", timeZone: \"UTC\" }),\n nf: (n) =>\n Intl.NumberFormat(undefined, {\n notation: \"compact\",\n compactDisplay: \"long\"\n }).format(n)\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":4530,"value":"numCounts = d3.rollups(\n wordleByHour,\n (d) => d3.sum(d, (t) => t.count),\n (d) => wordleNumToMainDate(d.wordle_num).toISOString()\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9735,"value":"### Tweet Volume Analysis","pinned":false,"hidden":false,"mode":"md"},{"id":4728,"value":"viewof bandwidth = Inputs.range([0, 1], { value: 0.16, step: 0.01 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":11499,"value":"{\n // by day of week\n Plot.plot({\n marks: [\n Plot.dot(volumeAnalysisData, {\n x: \"num\",\n y: \"logLoessResidualweekDayAdjusted\"\n }),\n Plot.link(volumeAnalysisData, {\n y1: (d) => d.weekDayMeanVolume,\n y2: (d) => d.weekDayMeanVolume,\n x1: d3.min(volumeAnalysisData, (d) => d.num),\n x2: d3.max(volumeAnalysisData, (d) => d.num),\n stroke: \"red\",\n strokeDasharray: [3, 2]\n }),\n Plot.frame(),\n Plot.ruleY([0])\n ],\n // marginTop: 100,\n fx: {\n domain: d3.range(7).map((d) =>\n new Date(new Date(0).setDate(d)).toLocaleDateString(\"en-US\", {\n weekday: \"long\"\n })\n )\n },\n x: { axis: null, inset: 10 },\n y: { domain: [-0.4, 0.4] },\n facet: { data: volumeAnalysisData, x: \"weekDay\" }\n });\n\n return \"volume viz\";\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":4090,"value":"// Plot.lineY(\n// d3.rollups(\n// wordleByHour,\n// (d) => d3.sum(d, (t) => t.count),\n// (d) => d.hour\n// ),\n// Plot.windowY(\n// { reduce: \"mean\", k: 24, anchor: \"middle\" },\n// {\n// x: (d) => new Date(d[0]),\n// y: (d) => d[1],\n// strokeWidth: 3,\n// stroke: \"forestgreen\"\n// // sort: \"hour\",\n// // title: \"wordle_num\"\n// }\n// )\n// ).plot({\n// marginLeft: 50,\n// width,\n// y: {\n// // domain: [0, d3.max(wordleByHour, (d) => d.count)],\n// domain: [0, 16000],\n// label: \"Wordle Tweets per Hour\",\n// grid: true\n// },\n// caption: \"Count by hour and wordle num\"\n// // style: {\n// // background: \"#383133\",\n// // color: \"white\"\n// // }\n// })\n\"count by hour and wordle num graph\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":7509,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs"},{"id":9333,"value":"### Wordle Difficulty Modeling Code","pinned":false,"hidden":false,"mode":"md"},{"id":11439,"value":"viewof bandwidth2 = Inputs.range([0, 1], { value: 0.16, step: 0.01 })","pinned":true,"hidden":false,"mode":"ojs"},{"id":11148,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs"},{"id":10707,"value":"regressionCombinations = getAllSubsets([...regressionPropList])\n .slice(1)\n .map((d) => ({\n factorNames: d.map((p) => p[0]),\n factorIndex: d3.sort(\n d.map((p) => [...regressionPropList.keys()].indexOf(p[0]))\n ),\n factorFields: d.map((p) => p[1]),\n model: olsModel(\n wordleDifficultyAnalysisData,\n \"meanScore\",\n d.map((p) => p[1])\n ),\n inverseModel:\n d.length !== [...regressionPropList].length &&\n olsModel(\n wordleDifficultyAnalysisData,\n \"meanScore\",\n [...regressionPropList.values()].filter(\n (p) => !d.map((p) => p[1]).includes(p)\n )\n )\n }))","pinned":false,"hidden":false,"mode":"ojs"},{"id":6825,"value":"wordleDifficultyResiduals = wordleDifficultyAnalysisData.map((d, i) => ({\n ...d,\n twoFactorResidual: twoFactorModel.resid[i],\n timelessResidual: fullFactorTimelessModel.resid[i],\n fullResidual: fullFactorTimeAwareModel.resid[i],\n customResidual: customDifficultyModel.resid[i]\n}))","pinned":false,"hidden":false,"mode":"ojs"},{"id":10415,"value":"customDifficultyModel = olsModel(\n wordleDifficultyAnalysisData,\n \"meanScore\",\n multiRegressionProps\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":9530,"value":"fullFactorTimeAwareModel = olsModel(wordleDifficultyAnalysisData, \"meanScore\", [\n \"harmonicMeanLetterFrequency\",\n \"uniqueLetterCount\",\n \"wordPrevalence\",\n \"num\",\n \"vowelStructureFrequency\",\n \"adjacentWordCount\"\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":10380,"value":"fullFactorTimelessModel = olsModel(wordleDifficultyAnalysisData, \"meanScore\", [\n \"harmonicMeanLetterFrequency\",\n \"uniqueLetterCount\",\n \"wordPrevalence\",\n \"vowelStructureFrequency\",\n \"adjacentWordCount\"\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":10361,"value":"twoFactorModel = olsModel(wordleDifficultyAnalysisData, \"meanScore\", [\n \"uniqueLetterCount\",\n \"harmonicMeanLetterFrequency\"\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":10431,"value":"regressionPropList = new Map([\n [\"Unique letter count\", \"uniqueLetterCount\"],\n [\"Letter rarity\", \"harmonicMeanLetterFrequency\"],\n [\"Word prevalence\", \"wordPrevalence\"],\n [\"Wordle date\", \"num\"],\n [\"Vowel structure frequency\", \"vowelStructureFrequency\"],\n [\"Adjacent word count\", \"adjacentWordCount\"]\n\n // [\"2nd Order Adjacent word count\", \"adjacent2WordCount\"],\n // [\"Adjacent answer word count\", \"adjacentAnswerWordCount\"],\n // [\"Adjacent filtered word count\", \"adjacentFilteredWordCount\"]\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":10348,"value":"olsModel = (data, dependentProp, independentProps) =>\n jstat.models.ols(\n data.map((d) => d[dependentProp]),\n data.map((d) => [1, ...independentProps.map((p) => d[p])])\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":6381,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs"},{"id":11318,"value":"wordleDifficultyOverTimeLOESS = d3\n .regressionLoess()\n .x((d) => d[0])\n .y((d) => d[1])\n .bandwidth(bandwidth2)(meanScoreByNum)","pinned":false,"hidden":false,"mode":"ojs"},{"id":8818,"value":"filterFn = (d, word) =>\n d\n .toLowerCase()\n .split(\"\")\n .filter((char) => /[aeiouy]/.test(char))\n .join(\"\") ===\n word\n .toLowerCase()\n .split(\"\")\n .filter((char) => /[aeiouy]/.test(char))\n .join(\"\") &&\n getHarmonicMeanLetterFreq(d) > getHarmonicMeanLetterFreq(word)\n// d","pinned":false,"hidden":false,"mode":"ojs"},{"id":3620,"value":"### Wordle Difficulty Viz","pinned":false,"hidden":false,"mode":"md"},{"id":2563,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs"},{"id":2390,"value":"topLabelPositions = [\n ...d3.cumsum(\n topRoundCounts.map((d) => d[1] / d3.sum(topRoundCounts, (d) => d[1]))\n )\n]\n .map((d, i, n) => (d + (n[i - 1] || 0)) / 2)\n .map((d, i) =>\n width < 500 ? (i == 0 ? d - 0.01 : i == 1 ? d + 0.01 : d) : d\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":2361,"value":"topRoundCounts = countByRound(\n wordleNumSummary.filter(\n (d) => d.wordle_num == sortedMeanScoreByNum.slice(-1)[0][0]\n )\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":8214,"value":"sortedMeanScoreByNum = d3.sort(meanScoreByNum, (d) =>\n sort == \"Difficulty\" ? d[1] : d[0]\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2346,"value":"meanScoreByNum = d3.rollups(\n wordleNumSummary,\n (d) => weightedMean(countByRound(d)),\n (d) => d.wordle_num\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":6186,"value":"### Word Property Analysis Functions","pinned":false,"hidden":false,"mode":"md"},{"id":6452,"value":"getAdjacentWordCount = (\n word,\n steps = 1,\n corpus = \"ALL\" /* ALL or ANS */,\n filter = (d) => d\n) => {\n const adjWords = getAdjacentWords(word, steps, corpus).filter((d) =>\n filter(d, word)\n );\n return adjWords.length;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":8752,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs"},{"id":6243,"value":"getUniqueLetters = (word) => +[...new Set([...word.toUpperCase()])].length","pinned":false,"hidden":false,"mode":"ojs"},{"id":6421,"value":"getWordLogFrequency = (word) => Math.log(wordFrequencies[word.toLowerCase()])","pinned":false,"hidden":false,"mode":"ojs"},{"id":10008,"value":"getWordPrevalence = (word) =>\n +prevalenceData.find((p) => p.Word == word.toLowerCase()).Prevalence","pinned":false,"hidden":false,"mode":"ojs"},{"id":6300,"value":"getVowelStructureFrequency = (word) =>\n structureFrequencies.get(getVowelStructure(word))","pinned":false,"hidden":false,"mode":"ojs"},{"id":6330,"value":"structureFrequencies = d3.rollup(\n words.answers,\n (d) => d.length,\n (d) => getVowelStructure(d)\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":6343,"value":"getVowelStructure = (word) =>\n word\n .toUpperCase()\n .replaceAll(/[AEIOU]/g, \"v\")\n .replaceAll(/[A-Z]/g, \"c\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":6279,"value":"getHarmonicMeanLetterFreq = (word) =>\n 1 /\n d3.mean(\n [...word.toUpperCase()].map(\n (d) => 1 / alphabet.find((l) => l.letter == d).frequency\n )\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":6274,"value":"getMedianLetterFreq = (word) =>\n d3.median(\n [...word.toUpperCase()].map(\n (d) => alphabet.find((l) => l.letter == d).frequency\n )\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":6192,"value":"getMeanLetterFreq = (word) =>\n d3.mean(\n [...word.toUpperCase()].map(\n (d) => alphabet.find((l) => l.letter == d).frequency\n )\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":6208,"value":"getMeanInverseLetterFreq = (word) =>\n d3.mean(\n [...word.toUpperCase()].map(\n (d) => 1 / alphabet.find((l) => l.letter == d).frequency\n )\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":7276,"value":"### Consistency Analysis Utils","pinned":false,"hidden":false,"mode":"md"},{"id":8991,"value":"$40","pinned":false,"hidden":false,"mode":"ojs"},{"id":3198,"value":"### Grid Rendering","pinned":false,"hidden":false,"mode":"md"},{"id":3996,"value":"viewof startPoint = Scrubber(d3.range((matrixArtData.length - 15) * 5), {\n delay: 1000 / 6\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":3307,"value":"showSVGMosaic = (data, gameRows, gameCols) => {\n const cellSize = 20,\n gameWidth = cellSize * 5,\n gameHeight = cellSize * 6,\n _width = gameWidth * (gameCols - 1),\n height = gameHeight * gameRows;\n const svg = d3\n .create(\"svg\")\n .attr(\"viewBox\", [\n 0,\n 0,\n Math.floor((width * 1.1) / gameWidth) * gameWidth,\n height - (gameRows - 1) * cellSize\n ]);\n svg\n .selectAll(\"g\")\n .data(\n data.slice(\n Math.floor(startPoint / 5),\n Math.floor(startPoint / 5) + gameRows * gameCols\n )\n )\n .join(\"g\")\n .attr(\n \"transform\",\n (d, i) =>\n `translate(${[\n (i % gameCols) * gameWidth - (gameWidth / 5) * (startPoint % 5),\n Math.floor(i / gameCols) * cellSize\n ]})`\n )\n .each(function (d) {\n showSVGGrid(d, d3.select(this), gameRows);\n });\n return svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":312,"value":"showSVGGrid = (row, svg, gameRows) => {\n if (!svg) {\n svg = d3.create(\"svg\");\n }\n const cellSize = 20;\n const data = row.matrix;\n svg\n .selectAll(\"text\")\n .data(data)\n .join(\"text\")\n .attr(\"x\", (d, i) => (i % 5) * cellSize)\n .attr(\"y\", (d, i) => Math.floor(i / 5) * cellSize * gameRows + 15)\n .text((d) => toEmoji(d));\n return svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":783,"value":"toEmoji = (letters) =>\n letters.replaceAll(\"A\", \"⬜️\").replaceAll(\"B\", \"🟨\").replaceAll(\"C\", \"🟩\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":2906,"value":"### Util Functions","pinned":false,"hidden":false,"mode":"md"},{"id":6368,"value":"linRegPlot = (data, xProp, yProp, labelProp) => {\n const reg = linReg(\n data,\n (d) => d[xProp],\n (d) => d[yProp]\n );\n return Plot.plot({\n marks: [\n labelProp\n ? Plot.text(data, {\n x: xProp,\n y: yProp,\n text: labelProp\n })\n : Plot.dot(data, {\n x: xProp,\n y: yProp,\n r: 1\n }),\n Plot.line(reg, {\n x: (d) => d[0],\n y: (d) => d[1],\n stroke: \"red\",\n strokeDasharray: [3, 3]\n })\n ],\n inset: 10,\n caption: `a: ${reg.a.toLocaleString()}, \n b: ${reg.b.toLocaleString()}; \n r^2: ${reg.rSquared.toLocaleString()};\n p: ${reg.p2.toLocaleString()} (${\n reg.p2 > 0.05 ? \"ns\" : reg.p2 > 0.01 ? \"*\" : reg.p2 > 0.001 ? \"**\" : \"***\"\n })`\n });\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":4838,"value":"linReg = (data, x, y) => {\n const obj = d3.regressionLinear().x(x).y(y)(data);\n obj.n = data.length;\n obj.r = Math.sqrt(obj.rSquared);\n obj.t = (obj.r * Math.sqrt(obj.n - 2)) / Math.sqrt(1 - obj.r ** 2);\n obj.p1 = jstat.ttest(obj.t, obj.n - 2, 1);\n obj.p2 = jstat.ttest(obj.t, obj.n - 2, 2);\n return obj;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":10703,"value":"getAllSubsets = (theArray) =>\n theArray.reduce(\n (subsets, value) => subsets.concat(subsets.map((set) => [value, ...set])),\n [[]]\n )\n// https://stackoverflow.com/questions/42773836/how-to-find-all-subsets-of-a-set-in-javascript-powerset-of-array","pinned":false,"hidden":false,"mode":"ojs"},{"id":2338,"value":"weightedMean = (valueCountPairs) =>\n d3.sum(valueCountPairs, (t) => t[0] * t[1]) /\n d3.sum(valueCountPairs, (t) => t[1])","pinned":false,"hidden":false,"mode":"ojs"},{"id":2329,"value":"countByRound = (numRoundData) =>\n d3.rollups(\n numRoundData,\n (t) => d3.sum(t, (v) => v.count),\n (t) => (t.rounds == \"X\" ? 7 : +t.rounds)\n )","pinned":false,"hidden":false,"mode":"ojs"},{"id":1500,"value":"wordleNumToTimeRange = (num) => [\n new Date(new Date(\"2021-06-18\").setDate(num + 17)),\n new Date(new Date(\"2021-06-18\").setDate(num + 20))\n]","pinned":false,"hidden":false,"mode":"ojs"},{"id":2431,"value":"wordleNumToMainDate = (num) =>\n new Date(new Date(\"2021-06-18\").setDate(num + 18))","pinned":false,"hidden":false,"mode":"ojs"},{"id":394,"value":"getWord = (num) => words.answers[num]\n// ⛔️⛔️⛔️ INFO HAZARD ALERT ⛔️⛔️⛔️\n// Messing with this could get you information you don't want","pinned":false,"hidden":false,"mode":"ojs"},{"id":2114,"value":"surfaceCodes = [\n \"\",\n \"Twitter for iPhone\",\n \"Twitter for Android\",\n \"Twitter Web App\",\n \"Twitter for iPad\",\n \"Tweetbot for iΟS\",\n \"TweetDeck\",\n \"Other\"\n]","pinned":false,"hidden":false,"mode":"ojs"},{"id":2883,"value":"## Files & Imports","pinned":false,"hidden":false,"mode":"md"},{"id":2895,"value":"### Data files","pinned":false,"hidden":false,"mode":"md"},{"id":8038,"value":"matrixArtData = FileAttachment(\"matrixArtData.csv\").csv()","pinned":false,"hidden":false,"mode":"ojs"},{"id":8061,"value":"wordleByHour = FileAttachment(\"wordleByHour (4).csv\").csv({ typed: true })","pinned":false,"hidden":false,"mode":"ojs"},{"id":9284,"value":"prevalenceData = FileAttachment(\"fiveLetterPrevalenceData.csv\").csv()","pinned":false,"hidden":false,"mode":"ojs"},{"id":8048,"value":"wordleNumSummary = FileAttachment(\"wordleNumSummary (4).csv\").csv({\n typed: true\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":533,"value":"### Imports","pinned":false,"hidden":false,"mode":"md"},{"id":6417,"value":"wordFrequencies = d3.json(\n \"https://raw.githubusercontent.com/3b1b/videos/master/_2022/wordle/data/freq_map.json\"\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":4454,"value":"d3 = require(\"d3\", \"d3-regression@1\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":6007,"value":"jstat = require(\"jstat\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":11483,"value":"### Observable imports","pinned":false,"hidden":false,"mode":"md"},{"id":8503,"value":"import { rangeSlider } from \"@mootari/range-slider@1312\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":6178,"value":"import { alphabet } from \"@d3/bar-chart\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":5823,"value":"import { footnote } from \"@jashkenas/markdown-style-footnotes\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":4928,"value":"import { tweet } from \"@observablehq/tweet\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":3998,"value":"import { Scrubber } from \"@mbostock/scrubber\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":381,"value":"import { words } from \"@visnup/cheat-at-wordle\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":10164,"value":"import { footer } from \"@rlesser/footer\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":10166,"value":"footer(\"wordle-twitter-exploration\")","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W41","stdlib":"1","titling":"auto"}