31:T5db,md`# As votes come in, what would it take for the trailing candidate to win? There are two kinds of election forecast. The first uses proxies for the real vote: in the months before an election, polls and economic indicators and other signals point to how things may go before a single vote is cast. The second uses partial fragments of the real vote: on election night, as votes are counted and results come in, the media tries to predict whether the count so far, plus what’s known about the places yet to count, are sufficient to call the outcome. This year, the surge in mail and absentee ballots means we may not know who won by the end of the night, as Nick Corasaniti and Denise Lu describe in their excellent [timeline of when states process ballots](https://www.nytimes.com/interactive/2020/10/13/us/politics/when-votes-counted.html) in The New York Times. The second forecast stage, trying to figure out what happened based on incomplete information, may be particularly prolonged and agonizing. The interpretation of partial results will vary state to state; it will depend on the biases inherent in different voting methods in different places. This notebook doesn’t talk about any of that. It just focuses on how to look at the simplified arithmetic of certain questions we ask ourselves: Can a lead hold? Can it be surmounted? Around an election you see results reported like this, with votes cast so far and some indication (the gray footer) of how many remain to be counted:`32:T517,md`This is just arithmetic, not statistics or probability. The sum of your votes so far and the votes you’ll get must be more than ${pct( winThreshold )}. It’s _possible_ that if you’re losing by a lot, you could make up for it by winning the remainder by a lot. In practice elections are tight and the path to victory is far narrower, because it’s unlikely that your remaining results will be so different from your results so far. We can use this two-dimensional space as a backdrop upon which to draw the path elections might take as different precincts report results. An election starts at the top, with no votes counted; then the earliest votes come in, while the totals are low and swings are easy; then it settles down and converges on its final result. This time, as you drag the circle along some path from top to bottom, you’re only able to make moves that would actually be possible, as shown by the black dashed cone. (If you’re tied at 5 votes to 5, and count another 10 votes, the race could go to 5–15 or 15–5 or anywhere in-between, but neither side could possibly make it to 90% of the vote.) The path the circle takes from top to bottom is recorded by the white line on the right, atop contours representing how much of the remaining votes each party has to win: `33:T15fd,md`Again, I have not been able to find data on when different precincts reported which votes, so this can only be a very rough upper and lower bound on how it could go. Rural precincts may report faster than urban ones, which could lead to systematic bias. But I cannot quantify it any better here. We don’t know in what order the votes will come. There are no predictions here. This cannot tell you the chances that a given race at a given stage has such-and-such a chance of going the other way. It is a toy model for thinking about the basics of partial results, and makes many simplifications. Absentee ballots do not go through precincts. Precincts may not report their final number all at once. There are often more than two candidates. Different people vote every time, so you can never tell how many votes are outstanding. Only once you count 100% of anything can you point to when you were at 50%. The real story is in the concrete reality of uncounted votes in real places, not in this abstract math. I want to do more, and look forward to seeing what newsrooms do. But, for all this, the simple and familiar table at the top of this notebook remains a fine way to show results, so long as we interpret it with due mistrust honed by years of betrayals. For more on forecasting the outcome of an election from early results (and from pre-election polls), some papers I have skimmed: - 1953: [UNIVAC on Election Night](https://sci-hub.se/https://doi.org/10.1109/EE.1953.6438592) by A. F. Draper — UNIVAC correctly predicted an Eisenhower landslide at 8:30 p.m. on election night. “Having computed all the probabilities by states, these were then assembled into a 531-degree polynomial which was solved to get the probability distributed. Of course, 531 is the number of electoral votes…. The whole run on the machine from entry of data to final prediction took about 6 minutes.” - 1963: [An Application of ‘Post Stratification’ Technique in Local TV Election Predictions](https://www.jstor.org/stable/2747122) by Robert W. Clyde, William J. Hemmerle and T. A. Bancroft. - 1966: [A Generalized Technique for Predicting Election Results From Early Returns](https://www.jstor.org/stable/2747141) by J. H. Sedransk and Robert W. Clyde. - 2004: [Can A Voter In New York Make A Candidate Lose in California? An Experimental Test of the Release of Early Election Results On Voter Turnout](https://www.researchgate.net/publication/252977482_Can_A_Voter_In_New_York_Make_A_Candidate_Lose_in_California_An_Experimental_Test_of_the_Release_of_Early_Election_Results_On_Voter_Turnout) by Geoffrey D. Peterson — “The evidence clearly shows that access to early election returns can decrease voter turnout and impact West Coast elections…. Given this, society must seriously consider which is truly more important: freedom of the press, or the principle of one person, one vote.” - 2010: [What Do We Know at 7 PM on Election Night?](http://www.stat.columbia.edu/~gelman/research/published/electionnight4.pdf) by Andrew Gelman and Nate Silver — models based on the polls and early results did very well in 2008. - 2013: [Memories of Election Night Predictions Past: Psephologists and Statisticians at Work](https://doi.org/10.1080/09332480.2007.10722868) by Stephen E. Fienberg — covers John Tukey and Warren Mitofsky’s forecasting work for the major television networks in the 1960s. - 2017: [19 things we learned from the 2016 election](http://www.stat.columbia.edu/~gelman/research/published/what_learned_in_2016_5.pdf) by Andrew Gelman and Julia Azari — “It’s harder and harder to reach a representative sample of voters, and it’s been argued that much of the swing in the polls is attributable not to people changing their vote intention, but to changes in who responds or doesn’t respond.” - 2017: [Election Predictions as Martingales: An Arbitrage Approach](https://arxiv.org/pdf/1703.06351.pdf) by Nassim Nicholas Taleb — more focused on the lead-up than election night; a potent warning that I should not try to do statistics. - 2020: [Predicting Elections using Live Data at The Washington Post](https://cpb-us-w2.wpmucdn.com/express.northeastern.edu/dist/d/53/files/2020/02/CJ_2020_paper_27.pdf) by Leonard Bronner, Al Johri, and Jeremy Bowers — extrapolates from earliest results based on those precincts’ covariance with others in past elections, with extensive helpful references. “Precincts that came in early are generally more rural, while the later precincts are urban.” [Blog post](https://washpost.engineering/2019/11/01/how-the-washington-post-estimates-turnout-on-election-nights/), [plans for November](https://www.axios.com/washington-post-2020-election-model-972a8585-d1e0-4fb0-a2bf-8a97e1a9c218.html). - 2020: [An Updated Dynamic Bayesian Forecasting Model for the 2020 Election](http://www.stat.columbia.edu/~gelman/research/unpublished/hdsr_forecasting.pdf) by Merlin Heidemanns, Andrew Gelman, and G. Elliott Morris — focused on polls, not partial results; tries to adjust for nonresponse bias and emphasizes how states move together; also would have predicted a Clinton win in 2016, but not so overconfidently. [Forecast](https://projects.economist.com/us-2020-forecast/president), [explanation](https://projects.economist.com/us-2020-forecast/president/how-this-works), [code](https://github.com/TheEconomist/us-potus-model). See also Isaac Asimov’s short story, “[Franchise](https://archive.org/details/1955-08_IF/page/n3/mode/2up)” (1955). Thanks to Kerry Rodden, Ian Johnson, Shan Carter, and Claire Collery for their feedback.`34:T9ca,md`Let’s use these variables: ${tex.block`\begin{aligned} x &: \text{fraction of current votes for Party A} \\ y &: \text{fraction of total votes counted} \\ z &: \text{fraction of remaining votes needed to win} \\ t &: \text{fraction needed to win} \end{aligned}`} The intuitive way to phrase our equation, our constraint, is that you win if the sum of the votes you’ve gotten so far and the votes you’ll get is greater than the win threshold: ${tex.block`xy + z(1 - y) \gt t`} Solve for ${tex`z`}: ${tex.block`z \gt \frac{xy - t}{y - 1}`} Now we can plug in our current share of votes and share of votes counted and find how much of the remaining votes we need: ${tex.block`z \gt \frac{${chart.x.toFixed(2)} \times ${chart.y.toFixed( 2 )} - ${winThreshold}}{${chart.y.toFixed(2)} - 1} = ${chart.z.toFixed(2)}`} I guess we could ([use WolframAlpha to](https://www.wolframalpha.com/input/?i=d%2Fdx+%28x+y+-+t%29%2F%28y+-+1%29)) take the partial derivatives with respect to ${tex`x`} and ${tex`y`} to see how the hurdle to win changes as the balance shifts and as votes come in: ${tex.block`\begin{aligned} \frac{\partial z}{\partial x} &= \frac{y}{y - 1} \\ \frac{\partial z}{\partial y} &= \frac{t - x}{(y - 1)^2} \end{aligned}`} So, holding ${tex`y`} fixed, share remaining is linear with respect to ${tex`x`}. It’s in ${tex`y`} that it’s nonlinear. And both blow up as ${tex`y`} goes to ${tex`1`}. When I bring back the interactive circle-dragging graphic to introduce the colored contours, I show a black dashed line indicating where you can drag the circle. Given the current ${tex`x_1`} and ${tex`y_1`}, and dragging down to a new level ${tex`y_2`}, what are the possible values of ${tex`x_2`}? I compute it by imagining that all the new votes between ${tex`y_1`} and ${tex`y_2`} (i.e. ${tex`y_2 - y_1`} of the total) goes to ${party( "B" )} (the leftmost bound), or to ${party( "A" )} (the rightmost bound). That comes out to: ${tex.block`x_2 \in \left[ \frac{x_1 y_1}{y_2}, \frac{x_1 y_2 + (y_2 - y_1)}{y_2} \right]`} I can’t quite wrap my head around the formal relationship between this and the equations above. Dragging through the cone and watching the contours, it is striking how the cone of possible moves hugs the curve of the victory threshold. Try going right up to the bottom right victory corner and then pivot to go left as fast as possible. It feels good, like discovering one curve in slices of the other. I feel like I’m missing something obvious.`35:[] 2f:{"title":"As votes come in, what would it take for the trailing candidate to win?","cells":[{"id":702,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":634,"value":"fig1 = table","pinned":false,"hidden":false,"mode":"ojs"},{"id":0,"value":"md`“Precincts reporting” doesn’t mean percent of votes counted, but for now we’ll pretend it does. Progress plays out in two dimensions: share of votes won so far, and share of total votes counted. The bigger your lead, the harder it is for the other side to overcome. The more votes have been counted, the harder a given lead is to overcome.\n\nDrag the circle in the middle of the chart below to set the percentage of votes so far won by your party (horizontal) and percentage of votes counted (vertical).\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":2,"value":"fig2 = viewof chart","pinned":false,"hidden":false,"mode":"ojs"},{"id":581,"value":"md`We can see all possibilities at once by laying out many of these charts, each as if the circle were dragged to a different spot. From left to right, ${party(\n \"A\"\n)} has more of the counted votes. From top to bottom, more of the votes have been counted.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":440,"value":"fig3 = minis","pinned":false,"hidden":false,"mode":"ojs"},{"id":584,"value":"md`If you squint you might see a kind of funnel: from top to bottom there is less and less uncertainty. All across the top, the necessary share of remaining votes is a toss-up, 50%; as you go down, that hurdle rises for the losing side until it becomes more than 100%, and thus insurmountable. Victory lies in the bottom corners: ${party(\n \"A\"\n)} wins in the bottom right, and ${party(\n \"B\"\n)} in the bottom left; eventually you come too close to one corner or the other, and it’s over.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1518,"value":"viewof pov = {\n const node = html``\n node.addEventListener(\"input\", () => node.value = node.elements[\"pov\"].value)\n node.value = \"A\"\n return node\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":463,"value":"fig4 = outcomes","pinned":false,"hidden":false,"mode":"ojs"},{"id":931,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":1319,"value":"fig5 = viewof dualChart","pinned":false,"hidden":false,"mode":"ojs"},{"id":1327,"value":"md`The contours on the right introduce a new color scheme that’s sort of flipped compared to the rectangles on the left, showing closeness-to-winning instead of a vote sum. ${party(\n \"A\"\n)} has been on the left in the rectangles, but it increases by going to the _right_, and wins when it makes it far enough right — so it’s the contours on the _right side_ that have its colors.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":934,"value":"fig6 = bilateralLegend","pinned":false,"hidden":false,"mode":"ojs"},{"id":1318,"value":"md`Now we can introduce some data. The paths below use the vote tallies of the precincts of ${county} County, Florida, in the 2016 presidential election. In this case, ${party(\n \"A\"\n)} is Donald Trump’s ticket. The different lines show how the vote could have developed over the course of the night, depending on the order in which precincts reported. For the smooth outer lines, the precincts are sorted by which way they lean. For the jagged lines in the middle, they report randomly. Each precinct’s vote tally is real, but I could not find the real order.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":909,"value":"fig7 = contours","pinned":false,"hidden":false,"mode":"ojs"},{"id":913,"value":"fig8 = outcome","pinned":false,"hidden":false,"mode":"ojs"},{"id":1137,"value":"md`There are many possible paths, but, if every vote is counted, they lead to the same result. If there’s no correlation between how fast a precinct counts and which candidate it favors, then the path will run down the cone’s middle. But if precincts favoring one party count fastest, then the results will begin on that side and then move toward the center.\n\nIf counting were stopped early, then bias in the order could be decisive.\n\nYou can select a different Florida county below to show above; each shows its cone of the most biased possible sequences of precincts reporting. Look at how far down the coloring starts on each side of the ⋁; it shows the earliest and latest that the final result might be attained.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1091,"value":"fig9 = viewof county","pinned":false,"hidden":false,"mode":"ojs"},{"id":972,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":539,"value":"md`---\n\n## Appendix`","pinned":false,"hidden":false,"mode":"ojs"},{"id":777,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":292,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":137,"value":"// currentShare ∈ [0,1] fraction of votes so far cast for Party A\n// votesCounted ∈ [0,1] fraction of total votes that have been counted\n// shareNeeded ∈ [0,1] fraction of remaining votes needed for Party A to win\nshareNeeded = (currentShare, votesCounted) =>\n votesCounted === 1\n ? currentShare > .5\n ? -Infinity\n : currentShare < .5\n ? Infinity\n : NaN\n : (currentShare * votesCounted - winThreshold) / (votesCounted - 1)","pinned":true,"hidden":false,"mode":"ojs"},{"id":419,"value":"winThreshold = 0.5","pinned":true,"hidden":false,"mode":"ojs"},{"id":127,"value":"hue = Math.floor(Math.random() * 360)","pinned":true,"hidden":false,"mode":"ojs"},{"id":90,"value":"color1 = `hsl(${hue}, 50%, 50%)`","pinned":true,"hidden":false,"mode":"ojs"},{"id":92,"value":"color2 = `hsl(${(hue + 180) % 360}, 50%, 50%)`","pinned":true,"hidden":false,"mode":"ojs"},{"id":1432,"value":"party = (letter = \"A\") =>\n html`PARTY ${letter}`","pinned":true,"hidden":false,"mode":"ojs"},{"id":1194,"value":"md`---\n\n## Components`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1172,"value":"// Simple table of results at top\nimport { table, pct } with { color1, color2, chart } from \"8ed8fae68b806147\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1160,"value":"// 2D draggable phase space\nimport { viewof chart } with { color1, color2 } from \"7dde4d42ff1f3a76\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1162,"value":"// Small multiples of phase space\nimport { minis } with { color1, color2, chart } from \"6e7b258359052066\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1167,"value":"// Small multiples of phase space, highlighting share needed to win and outcome\nimport { outcomes } with { color1, color2, chart, pov } from \"fb29011ee10bdeee\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1315,"value":"// Dual phase space introducing contours with cone of possible moves\nimport { viewof dualChart } with { color1, color2 } from \"9ecea15c8d9a9f1f\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":907,"value":"// Contour plot with semi-real historical paths drawn atop\nimport { chart as contours, outcome, bilateralLegend } with {\n color1,\n color2,\n shareNeeded as value,\n county\n} from \"1549aa94f8c6b9d1\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1089,"value":"// Final small multiples with each county’s cone\nimport { viewof county } with { color1, color2 } from \"16ab9fc154f01cc1\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":755,"value":"md`---\n\n

**Future work:** I would love to get real historical data for what vote was reported at what time, so as to plot it and then make a heatmap of the fraction of time a lead at certain stages of the phase space was overcome. (With this percentage of votes reported and this share of votes, what percentage of the time has it been overcome?) Also, I would like to clarify “the earliest and latest that the final result might be attained”. Also, some animations should only be triggered when they scroll into view. More generally, I would like to try to visualize the concrete situation on the ground instead of mere arithmetic.

`","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W35","stdlib":"1","titling":"auto"}