31:T6d3,md`There is an interactive playground below - don't miss it! In the last few weeks since the SARS COV 2 virus that causes COVID-19 (aka the Coronavirus) has spread I have repeatedly heard some usually well informed journalists express outrage that testing is conditional on very stringent criteria and not available to anyone who thinks they might have the virus. While it is true that we should improve our testing capacity and to some degree broaden the application of tests, I think that not enough awareness exists that testing always has downsides as well and that widespread testing on rare diseases can lead to very **counter-intuitive results**. Specifically, if the condition is rare and you test people randomly (or the entire population, doesn't matter) with a test that is not 100% perfect and thus has some false positives, then the majority of people tested *positive* will in fact not have the condition! This is a result of what is called [**Bayes' Theorem**](https://en.wikipedia.org/wiki/Bayes%27_theorem). This is a potentially big problem if a positive test e.g. means that a person (and maybe even their immediate family) must stay home for two weeks after a positive test. This is all very abstract and so below I created a simple interactive playground that you can use to calculate what happens if a test is applied entirely randomly. Drag the sliders and see what happens! One last note before we jump into the weeds: This is *not specific* to the current SARS-COV-2 tests but a general phenomenon that I think is good to know about. I am also not an Epidemiologist so take everything I say with a grain of salt :). Nevertheless, some thoughts on how this might apply to the current SARS-COV-2 are at the bottom.` 32:T745,md`#### Interpreting the above numbers If this is still confusing to you, let me try to explain. In the above scenario, we have an advantage that you don't have in real life: we know the true number of infected people to be ${infected} (set by the slider above), which means that the rest of the population (${population - infected} people) is not infected. If we now test ${population * percentTested / 100.0} then among these there will be ${(infected * percentTested / 100.0)} infected and ${(population - infected) * percentTested / 100.0} non-infected. We can now look at four cases. Let's consider the people we know are infected first. Our test has a false negative rate of ${(falseNegativesPer1000/10.0)}% so of the ${(infected * percentTested / 100.0)} infected poeple we test, ${truePositives} will indeed test positive (these are the true-positives) and ${falseNegatives} will test negative (these are the false negatives). So far so expected. But now let's look at the rest of the population. Of the ${(population - infected) * percentTested / 100.0} people that are not infected but are tested (with a false positive test rate of ${falsePositivesPer1000/10.0}%), ${trueNegatives} will indeed also test negative (these are the true negatives), but ${falsePositives} will test positives (these are the false positives). As someone who gets the test, you don't know if you really have the condition or not - you just know the test result. We know that of the tested population there were ${truePositives + falsePositives} positive results (the sum of true positives and false positives), but of those ${falsePositives} where false positives and only ${truePositives} were true positives, which leads to the above probability of being truly positive if tested positive of only ${Math.round(probabilityOfBeingPositiveIfTestedPositive*100)/100}% `33:T73c,md`# What does it mean? Should we not test? Of course we should test - but I think it is important to understand that **even testing has downsides**. When the suspected numbers of cases is very low as it is now in many countries with SARS COV-2, it makes a lot of sense to make testing conditional to cases where you have a reason to suspect that the person might have the condition - for example they might have symptoms and they were in an area with higher incidence rates. In general, it is important to take other evidence into account. But since SARS COV-2 seems to often lead to only mild, common cold like symptoms and with many people now being worried and probably wanting to be tested, it's important to see that just testing for the sake of it without additional evidence might have more negative public health effects than positives. As seen above, making sure that you test again if you recieve a positive result is an excellent way of combating the problem, but depending on the false positive rate and how rare the condition is the ratio of false positives to true positives can still be quite high even with two stage testing. It is also important to understand that people take medical tests in general to be less fallible than they are and with more testing you risk more people getting false negatives and falsly thinking that they can't be a danger to others. It is also important to consider that with such a new condition, testing capacity is limited and it makes sense to use it where it can do the most good. We should build out the capacity to test more and gradually loosen the conditions for testing, but at the same time it is important to communicate that any kind of testing has false results and are merely evidence that should help us update our believe in one direction or the other, not taken as absolute truth.`34:T484,md `## What does this mean for the current COVID-19 case numbers? As of writing in early March 2020, case numbers outside of China are still quite low. Do these numbers thus include a lot of false positives already? Probably not - current guidelines advise testing only for cases where other indications suggest a possible infection and people are apparently often confirmed with other methods. At least in many European labs the practice with the most common lab test (a Reverse transcription polymerase chain reaction assay) seems to be to only consider a case positive [if two labs give a positive result](https://www.rki.de/DE/Content/InfAZ/M/MERS_Coronavirus/MERS-CoV_Labordiagnostik.html;jsessionid=6C72BF7CAD26F01FC339E0CF6C62098D.internet062#doc3241470bodyText4). These are excellent ways of combating the above problem. What I wanted to show is merely that the understandable impulse of "we should just test everyone/most people/anyone who wants to be tested - is problematic, and that testing capacity, specificity of the tests, expected case numbers etc all have to be carefully evaluated together to arrive at a reasonable testing strategy.`35:[] 2f:{"title":"Should we test all people for the SARS COV 2 Virus if we could?","cells":[{"id":0,"value":"md`# Should we test all people for the SARS COV 2 Virus if we could?`","pinned":false,"hidden":false,"mode":"ojs"},{"id":54,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":80,"value":"md`### Population size\nI set this to roughly the size of Germany but feel free to change it by clicking on the left and updating the code below (everything else will update automatically)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":19,"value":"population = 80000000","pinned":false,"hidden":false,"mode":"ojs"},{"id":70,"value":"md`### Number of infected people\n\nDrag this slider to see how the ratio of false positives to true positives gets better as the condition gets more widespread`","pinned":false,"hidden":false,"mode":"ojs"},{"id":4,"value":"viewof infected = slider({\n min: 1000, \n max: 500000,\n value: 10000,\n step: 1000,\n description: \"Infected people\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":76,"value":"md`### False positive rate\n(This is the inverse of what is often called sensitivity). 10 false positives in 1000 is usually considered to be very good already with many tests having more false positives than that.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"viewof falsePositivesPer1000 = slider({\n min: 1, \n max: 100, \n step: 1,\n value: 10,\n description: \"False positive rate (X in 1000 false positives)\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":133,"value":"md`# Results` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":118,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":145,"value":"md`### Wait - what?` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":85,"value":"md`This may seem weird, right? After all, we explicitly specified that the test is quite accurate with only a ${falsePositivesPer1000/10.0}% false positive rate and a ${falseNegativesPer1000/10.0}% false negative rate. Shouldn't the probability that you really have the condition be roughly like the inverse of one of the above numbers?\n\nTo understand this, it helps to look at the total numbers of test results in all 4 categories below (true positives, false positives, false negatives and true negatives). If the disease is rare and you test a lot of people, there will be far more false positives than there are true positives!\n\nYou can adjust the values below and see how everything changes`","pinned":false,"hidden":false,"mode":"ojs"},{"id":48,"value":"probabilityOfBeingPositiveIfTestedPositive = (truePositives / (truePositives + falsePositives)) * 100.0","pinned":false,"hidden":false,"mode":"ojs"},{"id":37,"value":"truePositives = Math.round( (infected * percentTested / 100.0) * (1.0 - (falseNegativesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":34,"value":"falsePositives = Math.round((((population - infected) * percentTested / 100.0) * (falsePositivesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":40,"value":"trueNegatives = Math.round(((population - infected) * percentTested / 100.0) * (1.0 - (falsePositivesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":21,"value":"falseNegatives = Math.round((infected * percentTested / 100.0) * (falseNegativesPer1000 / 1000))\n ","pinned":true,"hidden":false,"mode":"ojs"},{"id":181,"value":"viewof percentTested = slider({\n min: 1, \n max: 100,\n value: 100,\n step: 1,\n description: \"Percent of population tested\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":13,"value":"viewof falseNegativesPer1000 = slider({\n min: 1, \n max: 100, \n step: 1,\n value: 10,\n description: \"False negative rate (X in 1000 false negatives)\"\n}) // the inverse of what is called the tests specificity (but I think false negative rate is easier to understand\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":240,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":273,"value":"md `## Does it help if we test twice if positive?\n\nYes! If it's possible to perform another test (preferably by another lab etc) if you tested positive one time then this is an excellent way to reduce the above problem. Cases where two tests now have different results would be inconclusive and would have to run a third test to get a get a more reliable result. Here are the numbers for what happens when you always test positive results again to confirm:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":305,"value":"html`

If in a population of ${population/1000000} million people where ${infected} have the disease, and you test either the entire population or a truly random sample, and your test has a false positive rate of ${falsePositivesPer1000} in 1000, then if you are tested and get a positive result and this is then repeated by another lab and you get a positive result AGAIN, the probability that you really have the disease based on these two tests alone is ${probabilityOfBeingPositiveIfTestedPositiveTwice.toPrecision(2)}%. This is much better than a single stage test (as seen above) which would result in a true positive probability of only ${probabilityOfBeingPositiveIfTestedPositive.toPrecision(2)}%

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":301,"value":"probabilityOfBeingPositiveIfTestedPositiveTwice = tp_second_test_positive / (fp_second_test_positive + tp_second_test_positive) * 100","pinned":true,"hidden":false,"mode":"ojs"},{"id":278,"value":"tp_second_test_positive = Math.round(truePositives * (1.0 - (falseNegativesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":283,"value":"tp_second_test_negative = Math.round(truePositives * (falseNegativesPer1000 / 1000))","pinned":true,"hidden":false,"mode":"ojs"},{"id":288,"value":"fp_second_test_positive = Math.round((falsePositives * (falsePositivesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":290,"value":"fp_second_test_negative = Math.round(falsePositives * (1.0 - (falsePositivesPer1000 / 1000)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":311,"value":"inconclusive_cases = tp_second_test_negative + fp_second_test_negative","pinned":true,"hidden":false,"mode":"ojs"},{"id":113,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":264,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":260,"value":"md `I hope this was useful! Let me know if you have any comments by tweeting at [@danyx23](https://twitter.com/danyx23)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":"import {slider} from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W35","stdlib":"1","titling":"auto"}