31:Tc18,md`# Estimating SARS-COV-2 infections ### from COVID-19 confirmed deaths This notebook tries to estimate how many **actual cases** of SARS-COV-2 infections there might be today and contrasts this with the number of **confirmed cases** known today (i.e. those that got a test and the test was positive). For example for ${country} the method described here estimates that there are ${(Math.round(targetCountryData[targetCountryData.length - 1][fields.projected], 0)).toLocaleString()} actual cases today when the official number of confirmed cases is ${targetCountryData[targetCountryData.length - 1][fields["total-cases"]].toLocaleString()} This estimation is based on an idea in Thomas Pueyo's medium post [Coronavirus - act today or people will die](https://medium.com/@tomaspueyo/coronavirus-act-today-or-people-will-die-f4d3d9cd99ca) and I want to stress that it is only a very rough estimate. Let me illustrate the basic idea with some simplified numbers: We know that for those people who die from COVID-19, it takes on average 23 days from infection to death. We also know that if we can guarantee good medical support, then on average only around 1 infected out of 100 will die (depending on various factors this might be higher or lower but 1 in 100 is a decent base for this explanation) Let's assume today is March 23rd and we have 2 known deaths in our country. By the above for there to be 2 deaths there must be 200 infected and, crucially, this number of infections must have happened around 23 days in the past, on March 1st. From this base of 200 infected on March 1st we can then calculate forward an exponential growth curve to see how many actual cases we expect there to be today. In the next section, "Prediction results" you can play interactively with several sliders to see what this looks like with concrete values and both the text and the graph below will update interactively. Further down you will find more information on data sources and the entire code that is behind these calculations. All of this is only a very crude way of approximating actual cases. The projection of actual cases only works well if a large fraction of COVID-19 deaths are diagnosed correctly (which is probably sort-of true for many high income countries but probably false for many poorer countries; however some serious under-reporting may happen in hotspots and some countries only report deaths from Hospitals - so the numbers of deaths here are probably only a lower bound). Still for all it's shortcomings, I find this simple estimation suprisingly informative. If you have any feedback, find something broken or if you have suggestions, please get in touch via [twitter](https://twitter.com/danyx23) or [email](mailto:obs@danyx.com) This notebook was written in late March 2020 when many countries in Europe started to severly restrict movement and in the original version it had two different growth rates where one could be used for the time before a lockdown and one after. In mid May 2020 this features was removed as it complicated the notebook unneccesarily. `32:T7e9,html`

Given that there are ${_.last(targetCountryData)[fields.deaths]} deaths today in ${country}, we calculate that ${averageDaysToDeathAfterInfection} days ago on ${latestDate.clone().subtract(averageDaysToDeathAfterInfection, "days").format("MMMM D")} there where ${Math.round(targetCountryData[targetCountryData.length - 1 - averageDaysToDeathAfterInfection][fields.projected]).toLocaleString()} actual cases. On this day, official reports confirmed ${targetCountryData[targetCountryData.length - 1 - averageDaysToDeathAfterInfection][fields.cases]} cases. (This is a large difference but remember that many of the actual cases we calculated would not have had symptoms yet, so a significant level of under-reporting at this point is to be expected).

If we calculate forward with a growth rate of after that we arrive at the following values for today:

Estimated actual case count today: ${(Math.round(targetCountryData[targetCountryData.length - 1][fields.projected], 0)).toLocaleString()}

Confirmed cases today: ${targetCountryData[targetCountryData.length - 1][fields["total-cases"]].toLocaleString()}

This means that the confirmed cases are ${(targetCountryData[targetCountryData.length - 1][fields.cases] / targetCountryData[targetCountryData.length - 1][fields.projected] * 100).toFixed(1)} % of what we estimate to be the true count of infected people today.

From todays estimated actual case count of ${(Math.round(targetCountryData[targetCountryData.length - 1][fields.projected], 0)).toLocaleString()}, with an infection fatality rate of ${infectedFatalityRate}% we thus expect that ${averageDaysToDeathAfterInfection} days in the future, i.e. on ${latestDate.clone().add(averageDaysToDeathAfterInfection, "days").format("MMMM D")} there will be around ${(Math.round(targetCountryData[targetCountryData.length - 1][fields.projected] * (infectedFatalityRate / 100), 0)).toLocaleString()} total deaths ` 33:Tc1f,chart = { const annotations = { "mark": { "type": "rect" }, "data": { "values": annotationPeriods }, "encoding": { "x": { "field": "start", "type": "temporal" }, "x2": { "field": "end", "type": "temporal" }, "opacity": { "value": 0.2 }, "color": { "field": "period" } } } const tooltip = [ { "title": "Total confirmed cases", "field": fields.casesFormatted, "type": "quantitative" }, { "title": "Total confirmed deaths", "field": fields["total-deaths"], "type": "quantitative" }, { "title": "Estimated actual cases", "field": fields.projectedFormatted }, { "title": "Date", "field": fields.date, "type": "temporal" } ] return vegalite({ "vconcat": [ { "layer": [ annotations, { "mark": { "type": "line" }, "encoding": { "x": { "field": fields.date, "type": "temporal" }, "y": { "field": fields["total-cases"], "type": "quantitative", "scale": { "type": "linear" } }, "color": { "value": "#8888dd" }, tooltip: tooltip } }, { "mark": { "type": "circle" }, "encoding": { "x": { "field": fields.date, "type": "temporal" }, "y": { "field": fields.projected, "type": "quantitative", "scale": { "type": "linear" } }, "color": { "value": "#44aaaa" }, tooltip: tooltip } } ], "height": graphHeight / 3 * 2, "width": graphWidth, "title": "Confirmed cases and projected true cases" }, { "layer": [ annotations, { "mark": { "type": "line" }, "encoding": { "x": { "field": fields.date, "type": "temporal" }, "y": { "field": fields["total-deaths"], "type": "quantitative", "scale": { "type": "linear" } }, "color": { "value": "#dd8888" }, tooltip: tooltip }, "height": graphHeight / 3, "width": graphWidth, "title": "Confirmed deaths" } ]}], "data": { "values": targetCountryData }, "title": country, "width": graphWidth, } ) }34:T6fb,md`## Additional notes For countries that haven't had any confirmed COVID-19 deaths it can't be used and the estimate is quite useless if a country does not have the capability to test deceased for COVID-19 at a large scale. ## Data sources The original data for confirmed cases and deaths for all countries is collected by the European Centre for Disease Prevention and Control which is then [processed and published daily in nicer form by the great team at Our World in Data ](https://ourworldindata.org/coronavirus-source-data). The state level data for Germany the [official data by the Robert Koch Institute](https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Fallzahlen.html) is used and can be found here). The state level data for the US is collected by the New York Times from various state level agencies. Since CSV formatted data (or HTML tables in the case of the RKI) is not as nice to process in Javascript as Json, the upstream data releases are [ingested daily using Github Actions](https://github.com/DouglasConnect/covid-19/) into [Edelweiss Data](https://saferworldbydesign.com/edelweissdata/), a scientific data management platform we develop at Edelweiss Connect. We have a separate website that describes how we use [EdelweissData for COVID-19 applications](https://douglasconnect.github.io/covid-19/). If you are interested in using our REST APIs, you can browse the datasources here: * [Our world in data dataset](https://edelweissdata.com/dataexplorer?dataset=b55b229d-6338-4e41-a507-0cf4d3297b54%3A1) * [Robert Koch Institut dataset](https://edelweissdata.com/dataexplorer?dataset=8dde2785-8a2a-4847-80b8-982a691954d6%3A1) * [New York Times dataset](https://edelweissdata.com/dataexplorer?dataset=72421b77-78e8-4fad-9e3b-33956585e515%3A1). ` 35:T6c1,dataSources = [ { title: "Country data compiled by the ECDPC ", notes: "Worldwide time series data by the European Center for Disease Prevention and Control, compiled by Our World in Data", url: "https://api.edelweissdata.com/datasets/b55b229d-6338-4e41-a507-0cf4d3297b54/versions/latest/data", fields: {"projected": "projected", "projectedFormatted": "projectedFormatted", "casesFormatted": "casesFormatted" }, metadataUrl: "https://api.edelweissdata.com/datasets/b55b229d-6338-4e41-a507-0cf4d3297b54/versions/latest/metadata", rowProcessingFunction: row => row.data, initialSelection: "Germany" }, { title: "German state data", notes: "Data by the Germany Robert Koch Insitute. ", url: "https://api.edelweissdata.com/datasets/8dde2785-8a2a-4847-80b8-982a691954d6/versions/latest/data", metadataUrl: "https://api.edelweissdata.com/datasets/8dde2785-8a2a-4847-80b8-982a691954d6/versions/latest/metadata", fields: { "projected": "projected", "projectedFormatted": "projectedFormatted", "casesFormatted": "casesFormatted" }, rowProcessingFunction: (row => row.data), initialSelection: "Berlin" }, { title: "US state data", notes: "State level data from The New York Times, based on reports from state and local health agencies.", url: "https://api.edelweissdata.com/datasets/72421b77-78e8-4fad-9e3b-33956585e515/versions/latest/data", metadataUrl: "https://api.edelweissdata.com/datasets/72421b77-78e8-4fad-9e3b-33956585e515/versions/latest/metadata", fields: {"projected": "projected", "projectedFormatted": "projectedFormatted", "casesFormatted": "casesFormatted" }, rowProcessingFunction: (row => row.data), initialSelection: "New York" }, ]36:T989,calculateAndMergeTrueProjectedCases = { return function(rows) { // filter out days before the graphStartDAte const filteredRows = _.filter(rows, row => moment(row[fields.date]).isBetween(graphStartDate, artificialTimeSeriesEnd, null, "[]")) console.log("filtered", filteredRows) const filteredStartDate = moment.max(moment(_.first(rows)[fields.date]), graphStartDate) const endDate = moment.max([moment(_.last(rows)[fields.date]), artificialTimeSeriesEnd]) const startDate = moment.min([filteredStartDate, moment(endDate).subtract(averageDaysToDeathAfterInfection, 'days')]) const numDays = endDate.diff(startDate, "days") const days = _.map(_.range(0, numDays + 1), diff => moment(startDate).add(diff, "days")) console.log("days", days) const dateIndexedFilteredRows = _.fromPairs(_.map(filteredRows, row => [(moment(row[fields.date]).format("YYYY-MM-DD")), row])) let projectedDays = [] let lastDaysCases = undefined for (const day of days) { const currentDateData = dateIndexedFilteredRows[day.format("YYYY-MM-DD")] const xDaysInFutureData = dateIndexedFilteredRows[moment(day).add(averageDaysToDeathAfterInfection, "days").format("YYYY-MM-DD")] if (xDaysInFutureData) { const projected = xDaysInFutureData[fields["total-deaths"]] / (infectedFatalityRate/100) lastDaysCases = projected if (currentDateData) { projectedDays.push(assemblyFinalDataRow(currentDateData, projected)) } else { projectedDays.push(assemblyFinalDataRow({[fields.date]: day.format("YYYY-MM-DD")}, projected)) } } else { if (lastDaysCases === undefined) { console.log("lastdayscases was not defined!") } else { // decide if we are before or after the lockdown and choose the correct doubling rate let doublingTime = doublingTimeBeforeLockdown let projected = lastDaysCases * 2**(1/doublingTime) lastDaysCases = projected if (currentDateData) { projectedDays.push(assemblyFinalDataRow(currentDateData, projected)) } else { projectedDays.push(assemblyFinalDataRow({[fields.date]: day.format("YYYY-MM-DD")}, projected)) } } } } console.log("projected", projectedDays) return projectedDays } }37:[] 2f:{"title":"Estimating SARS-COV-2 infections","cells":[{"id":0,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":1011,"value":"md`## Country selection`","pinned":false,"hidden":false,"mode":"ojs"},{"id":778,"value":"viewof selectedDataSourceIndex = \n select({title: \"Data source\", \n options: \n _.map(dataSources, (source, index) => ({label: source.title, value: index}) )})\n \n ","pinned":false,"hidden":false,"mode":"ojs"},{"id":874,"value":"md`${selectedDataSource.notes} The data includes cases and deaths reported until approximately: ${moment(metadata.estimatedReportingCutoff).format(\"LLL (UTC Z)\")}` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":347,"value":"viewof country = select({title: \"Region to display\", options: countries, value: selectedDataSource.initialSelection})","pinned":false,"hidden":false,"mode":"ojs"},{"id":546,"value":"md`# Prediction results`","pinned":false,"hidden":false,"mode":"ojs"},{"id":490,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":549,"value":"md`## Primary parameters\n\n**Play with these sliders and dropdowns** and the numbers in the text above and the graph below will change interactively`","pinned":false,"hidden":false,"mode":"ojs"},{"id":887,"value":"md`##### Growth rates` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":206,"value":"viewof dailyGrowthRateBeforeLockdown = slider({\n min: 0.25, \n max: 100,\n value: countryDailyDeathsGrowthRateAverageLastThreeDays ,\n step: 0.25,\n description: \"Daily growth rate of true cases before lockdown in %\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":721,"value":"md`The above growth rate results in a **doubling** of the number of actual cases **every ${Math.round(doublingTimeBeforeLockdown, 1).toLocaleString()} days**. The default value of the slider is set to the average growth rate of the number of confirmed deaths of the selected country over the last three days. For reference - over the last ${averageDaysToDeathAfterInfection} days the averaged daily growth rate of *confirmed cases* was ${Math.round(confirmedCasesGrowthRate, 2).toLocaleString()}%`","pinned":false,"hidden":false,"mode":"ojs"},{"id":575,"value":"html`

Graph of estimated actual vs confirmed cases

\n\n

In the graph below the blue line shows the confirmed cases, the red line are the confirmed deaths and the teal dots are the estimated actual cases. Hover over the dots or lines to see a tooltip with the actual values and change the primary or secondary parameters to see the graph change

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1072,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":553,"value":"md`## Secondary parameters` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":51,"value":"viewof infectedFatalityRate = slider({\n min: 0.1, \n max: 10,\n value: 0.7,\n step: 0.1,\n description: \"Infection fatality rate in %\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":682,"value":"md`This is the fraction of all SARS-COV-2 infected (including asymptomatic) that will ultimately die of COVID-19. We don't know this number, but various estimates put it in the 0.3-2.0 range as long as a regions health care system is not overloaded (if that happens as in Lombardia in late March 2020 then this number might end up being higher). \n\nThe IFR is probably also very age-dependent, so populations that are older probably have a higher IFR than those with a younger population. \n\n[This study in the Lancet](https://www.thelancet.com/journals/laninf/article/PIIS1473-3099%2820%2930243-7/fulltext) estimates that of all infected (including asymptomatic), the fatality rate is 0.7%.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":56,"value":"viewof averageDaysToDeathAfterInfection = slider({\n min: 5, \n max: 40,\n value: 23,\n step: 1,\n description: \"Average time from infection to death\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":661,"value":"md`Time from infection to death; [this study in the Lancet](https://www.thelancet.com/journals/laninf/article/PIIS1473-3099%2820%2930243-7/fulltext) estimates on average ~5 days from infection to symptoms and ~18 days from symptoms to death so we arrive at a default value of 23 days from infection to death` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":1161,"value":"viewof graphWidth = slider({\n min: 300, \n max: 1000,\n value: 600,\n step: 20,\n description: \"Width of graph\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":674,"value":"viewof graphHeight = slider({\n min: 300, \n max: 1000,\n value: 460,\n step: 20,\n description: \"Height of graph\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":372,"value":"viewof graphStartDateSlider = slider({\n min: 0, \n max: 50,\n value: 40,\n step: 1,\n display: value => moment(\"2020-01-01\").add(value, \"days\").format(\"YYYY-MM-DD\"),\n description: \"Date of graph start\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":850,"value":"viewof artificialTimeSeriesEndSlider = slider({\n min: 0, \n max: 50,\n value: 50,\n step: 1,\n display: value => moment(latestDate).subtract((50-value), \"days\").format(\"YYYY-MM-DD\"),\n description: \"Hide data after this date from the prediction model\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":894,"value":"md`This slider allows you to hide data from the model so that you can see what the prediction would have been like with yesterdays data, or data a week ago etc..` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":459,"value":"viewof logarithmic = checkbox({title: \"Log/Linear display (currently broken)\", options: [{value: \"log\", label: \"logarithmic\"}]})","pinned":false,"hidden":false,"mode":"ojs"},{"id":568,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":531,"value":"md`# Code\n\nBelow you find all the calculation used in the above text and graph. Feel free to change it to your desire!\n\nIf a cell is not expanded, click on the gray area on the left to show the code\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":913,"value":"md`### Data source setup` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":1205,"value":"projectedFields = ({ \"projected\": \"projected\", \"projectedFormatted\": \"projectedFormatted\", \"casesFormatted\": \"casesFormatted\" })","pinned":false,"hidden":false,"mode":"ojs"},{"id":768,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":783,"value":"selectedDataSource = dataSources[selectedDataSourceIndex]","pinned":true,"hidden":false,"mode":"ojs"},{"id":5,"value":"// fetch the data from the edelweiss REST api (latest will make sure we get the newest dataset, updated daily)\nraw_data = {\n const query = countryFilter\n const encodedQuery = JSON.stringify({condition: query})\n let url = new URL(selectedDataSource.url)\n url.searchParams.append('query', encodedQuery);\n const data = fetch(url)\n .then(response => {\n if (!response.ok) throw new Error(response.status);\n return response.json();})\n return data\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":1224,"value":"countryFilter = ({\n exactSearch: [ { column: [\"location\"]}, country ]\n})","pinned":true,"hidden":false,"mode":"ojs"},{"id":61,"value":"data = raw_data.results.map(selectedDataSource.rowProcessingFunction)","pinned":true,"hidden":false,"mode":"ojs"},{"id":960,"value":"metadata = fetch(selectedDataSource.metadataUrl)\n .then(response => {\n if (!response.ok) throw new Error(response.status);\n return response.json();})","pinned":true,"hidden":false,"mode":"ojs"},{"id":919,"value":"md`Show first data for for debugging purposes`","pinned":false,"hidden":false,"mode":"ojs"},{"id":68,"value":"// Show the first row for debugging purposes\ndata[0]","pinned":false,"hidden":false,"mode":"ojs"},{"id":926,"value":"md`### Main estimation function`","pinned":false,"hidden":false,"mode":"ojs"},{"id":182,"value":"$36","pinned":true,"hidden":false,"mode":"ojs"},{"id":1017,"value":"assemblyFinalDataRow = function(data, projected) {\n const formattedCases = _.isNil(data[fields.cases]) ? \"no data\" : data[fields.cases].toLocaleString()\n return {...data, [fields.projected]: projected, [fields.projectedFormatted]: (Math.round(projected, 0)).toLocaleString(), [fields.casesFormatted]: formattedCases}\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":911,"value":"md`### Derived parameters`","pinned":false,"hidden":false,"mode":"ojs"},{"id":432,"value":"// This is the data for the shaded areas in the graph\nannotationPeriods = [ { start: graphStartDate.format(), end: latestDate.clone().subtract(averageDaysToDeathAfterInfection, \"days\").format(), period: \"deaths 23 days later / IFR\" },\n {start: latestDate.clone().subtract(averageDaysToDeathAfterInfection, \"days\").format(), end: latestDate.format(), period: \"constant growth rate extrapolation\"}\n ]","pinned":false,"hidden":false,"mode":"ojs"},{"id":373,"value":"graphStartDate = moment(\"2020-01-01\").add(graphStartDateSlider, \"days\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":857,"value":"artificialTimeSeriesEnd = moment(latestDate).subtract((50-artificialTimeSeriesEndSlider), \"days\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":707,"value":"doublingTimeBeforeLockdown = Math.log(2)/Math.log(1+(dailyGrowthRateBeforeLockdown/100))","pinned":false,"hidden":false,"mode":"ojs"},{"id":752,"value":"fields = ({...metadata.columnNames,...projectedFields})","pinned":true,"hidden":false,"mode":"ojs"},{"id":89,"value":"countries = metadata.regions","pinned":true,"hidden":false,"mode":"ojs"},{"id":116,"value":"data_by_countries = _.groupBy(data, row => row[fields.region])","pinned":false,"hidden":false,"mode":"ojs"},{"id":410,"value":"// Pick the country selected in the dropdown and calculate the projection for it\ntargetCountryData = calculateAndMergeTrueProjectedCases(data_by_countries[country])","pinned":true,"hidden":false,"mode":"ojs"},{"id":416,"value":"latestDate = moment(_.chain(data_by_countries[country]).map(datum => datum[fields.date] ).max().value())","pinned":true,"hidden":false,"mode":"ojs"},{"id":938,"value":"countryDailyDeathsGrowthRateAverageLastThreeDays = \n { if (data_by_countries[country].length > 3) {\n const len = data_by_countries[country].length\n const offset = _.slice(data_by_countries[country], len-4, len-1)\n return _.chain(data_by_countries[country]).takeRight(3).zip(offset).map(pair => pair[0][fields[\"total-deaths\"]]/pair[1][fields[\"total-deaths\"]]*100 - 100).sum().value()/3\n }\n else\n return 15.0\n \n }","pinned":true,"hidden":false,"mode":"ojs"},{"id":1129,"value":"confirmedCasesGrowthRate = { const ratio = targetCountryData[targetCountryData.length-1][fields[\"total-cases\"]] / targetCountryData[targetCountryData.length-averageDaysToDeathAfterInfection-1][fields[\"total-cases\"]]\n return (Math.pow(ratio, 1/averageDaysToDeathAfterInfection)-1.0)*100\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":930,"value":"md`### Imports` ","pinned":true,"hidden":false,"mode":"ojs"},{"id":274,"value":"import {slider, select, checkbox} from \"@jashkenas/inputs\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":10,"value":"import { vl } from \"@vega/vega-lite-api\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":220,"value":"moment = require(\"moment\");","pinned":true,"hidden":false,"mode":"ojs"},{"id":985,"value":"import { inputsGroup } from \"@bumbeishvili/input-groups\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":84,"value":"_ = require(\"lodash\");\n","pinned":true,"hidden":false,"mode":"ojs"},{"id":1067,"value":"vegalite = require(\"@observablehq/vega-lite\")","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W37","stdlib":"1","titling":"auto"}