31:T74d,

Analysis details

We regularly download and curate whole-genome data from GISAID. These data are passed through a SARS-CoV-2 analysis pipeline, which has not been published as a standalone tool, but has been used in a number of publications, including [1,2,3]. All sequences are partitoned into three-month intervals (by sampling dates) and subjected to a series of selection analyses. This notebook presents a temporal evolutionary history of selected sites, as characterized by the [FEL method]. During a given time interval, a site can be

Positively selected: natural selection may be promoting diversification (changes to different amino-acids) at this site.
Negatively selected: natural selection may be suppressing amino-acid variation at this site; the site is therefore evolutionary conserved at the protein level.
Evolving neutrally: all other cases; this includes cases where the FEL method lacks statistical power.

When a site is identified as evolving non-neutrally, a p-value from the likelihood ratio test is used to assess significance. These are reported in the plot. Note that p-values are not directly comparable across timepoints, because of different sample sizes and other factors.

32:T648,main_view = vegalite({ "title": null, data: {values: forPlot}, transform: [{"calculate": "-log(datum.p)/log(10)", "as" : "psize"}], width : 700, layer: [ { "mark": {"type": "circle", "opacity" : 0.5, "stroke": "black", "strokeWidth": 1, size : 40, tooltip : true}, "encoding": { "x": {field: "country", type: "ordinal", scale : {"domain" : sorted_countries}, axis: {"title": "Country", titleFontSize : 14, grid : false, "labelFontSize" : 12, labelAngle : 45}}, tooltip : [ {"field": "loc", "type": "ordinal", "title": "Site"}, {"field": "to", "type": "temporal", "title": "Analysis date"}, {"field": "p", "type": "quantitative", "title": "P-value", "format" : ".2g"}, {"field" : "seqs", "type": "quantitative", "title": "Repr. sequences used in analysis"} ], "y": {"sort": {"field" : "sorter"}, "field": "label", "type": "nominal", axis: { "titleFontSize" : 14, "labelFontSize" : 14, "grid" : false, "title" : null, "labelFont" : "courier" }, }, "size" : {"field" : "psize", "type" : "quantitative", "legend" : { "tickCount" : 5, "title" : "-log10 (p)", "orient" : "top"} }, "color": { "condition": {"test": "datum['alpha'] > datum['beta']", "value": "darkblue"}, "value": "darkred" } } } ] })33:T7b6,preset_mapping = ({ "Empty list" : [], "501Y metasignature" : pad_array(["nsp2/85","nsp2/501","nsp3/183","nsp3/370","nsp3/837","nsp3/890","nsp6/107","nsp6/260","exonuclease/496","helicase/47","helicase/599","E/71","S/18","S/26","S/69","S/98","S/138","S/144","S/215","S/243","S/417","S/484","S/501","S/655","S/681","S/701","S/716","S/1027","S/1118","S/1176","S/1264","ORF3a/57","ORF3a/171","N/205","N/235"]), "alpha (B.1.1.7)" : pad_array (["913","ORF1a/1001","ORF1a/1708","5986","ORF1a/2230","ORF1a/3675","14676","15279","16176","S/69","S/144","S/501","S/570","S/681","S/716","S/982","S/1118","ORF8/27","ORF8/52","ORF8/73","N/3","N/235"]), "beta (B.1.351)" : pad_array (["nsp2/85","ORF1ab/1655","ORF1ab/3353","S/80","S/215","S/484","S/501","S/701","ORF3a/57","ORF3a/171","E/71","N/205","22280","11287"]), "gamma (P.1)" : pad_array (["733","2749","ORF1a/1118","ORF1a/1795","11287","12778","13860","S/20","S/26","S/417","S/484","S/501","S/1027","ORF8/92","N/80"]), "delta (B.1.617.2)" : pad_array (["S/19","S/142","S/452","S/478","S/681","S/950","ORF3a/26","M/82","ORF7a/82","ORF7a/120","N/63","N/203","N/377"]), "mu (B.1.621)" : pad_array (["3037","ORF1a/1055","ORF1a/1538","6037","ORF1a/3255","ORF1a/3729","13057","ORF1b/314","ORF1b/1342","18877","19035","20148","S/95","S/145","S/346","S/484","S/501","S/614","S/681","S/950","ORF3a/57","ORF8/11","ORF8/38","ORF8/67","N/205"]), "omicron (B.1.1.529)" : pad_array (["NSP3/38", "NSP3/1069", "NSP3/1266", "NSP3/1892","NSP4/492","3C/132","NSP6/189","RDRP/323","exonuclease/42","S/67","S/95","S/142","S/212","S/339","S/371","S/373","S/375","S/417","S/440","S/446","S/477","S/478","S/484","S/493","S/496","S/498","S/501","S/505","S/547","S/614","S/655","S/679","S/681","S/764","S/796","S/856","S/954","S/969","S/981","E/9","M/3","M/19","M/63","N/13","N/203","N/204"]), "Selected in ≥50% of countries" : pad_array(commonSites), //"Negatively selected recent 3 intervals" : pad_array(recent_site_listN) })34:[] 2f:{"title":"Evidence of natural selection history operating on SARS-CoV-2 genomes enabled by data from ","cells":[{"id":0,"value":"md`# Evidence of natural selection history operating on SARS-CoV-2 genomes enabled by data from $\"gisaid-logo\"$
`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1137,"value":"For a given set of SARS-CoV-2 genomic sites, see in **which countries** selection was operating on those sites. These analyses use data collected over the entire sampling period (from early 2020 to present time) and include ${_.size (countries)} with sufficient sampling[Download .csv data](https://github.com/spond/SARS-CoV-2-variation)","pinned":false,"hidden":false,"mode":"md"},{"id":1123,"value":"$31","pinned":false,"hidden":false,"mode":"html"},{"id":1192,"value":"viewof preset = select ({\n options : sitelist ? [\"Supplied as arguments\"] : _.keys (preset_mapping),\n description: \"Select one of the presets for sites to display\",\n value: \"omicron (B.1.1.529)\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1170,"value":"viewof sites = multiAutoSelect ({\n options : sitelist ? [] : _.uniq (_.map (selectable_sites, (d)=>{let p = d.split (\"/\"); return \"Codon \" + p[0] + \"/\" + Math.floor (p[1])})),\n title: \"Track the following mutations\",\n value : sitelist ? _.map (sitelist.split (/[ ,]/), (d)=>\"Site \" + d) : preset_mapping[preset]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1152,"value":"**Figure 1.** Geographic locatilization of **${sites.length}** selected sites. If a site was found to be positively (negatively) selected in a specific country, a bubble will be drawn at a corresponding point on the plot. The area of the bubble is scaled as -log₁₀ (p), where p is the p-value of the FEL likelihood ratio test. Larger bubbles correspond to smaller p-values; p-values are not directly comparable between different time windows and different genes due to differences in sample sizes and other factors. The x-axis shows individual countries.","pinned":false,"hidden":false,"mode":"md"},{"id":588,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":1712,"value":"**Table 1.** Counts of selected sites by gene / country. \n - _Positive_ : if checked show positively selected sites, otherwise negatively selected sites.\n - _Scaled_ : if checked, normalize raw counts of selected size by `(gene length, Kb) X (total internal tree branch, subs/site)`, otherwise show raw counts.","pinned":false,"hidden":false,"mode":"md"},{"id":1758,"value":"viewof search = Inputs.search(_.chain (table1data).map((d)=> {\n let pass1 = _.mapValues (d, (c)=> {\n if (_.isObject (c)) return c[table1key];\n return c;\n })\n let selector = ['Country'];\n selector.push (...table_gene_options);\n return _.pick (pass1,selector);\n}).value())","pinned":false,"hidden":false,"mode":"ojs"},{"id":1765,"value":"viewof table_view_options = Inputs.checkbox([\"Positive\",\"Scaled\"], {\"value\" : [\"Positive\", \"Scaled\"], label: \"\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1796,"value":"viewof table_gene_options = Inputs.checkbox(gene_list, {\"value\" : gene_list, label: \"\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1762,"value":"viewof avtable = Inputs.table(search, {\n rows : 20,\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1812,"value":"**Figure 2.** A comparison of selected countries and genes","pinned":false,"hidden":false,"mode":"md"},{"id":1816,"value":"fig2view = vegalite({\n \"title\": null, \n data: {values: avtable_plot},\n width : 700,\n height : 300,\n \"mark\": {\"type\": \"circle\", \"opacity\" : 0.5, \"stroke\": \"black\",\n \"strokeWidth\": 1, size : 100, tooltip : true},\n \"encoding\": {\n \"x\": {field: \"Gene\", type: \"nominal\", axis: {\"title\": \"\", titleFontSize : 14, grid : false, \"labelFontSize\" : 12, labelAngle : 45}, \"scale\" : {\"domain\" : _.intersection(gene_list, table_gene_options)}},\n \n \n \"y\": {\"field\": \"Value\", \"type\": \"quantitative\", axis: {\n \"titleFontSize\" : 14,\n \"labelFontSize\" : 14,\n \"grid\" : false,\n \"title\" : null,\n \"labelFont\" : \"courier\" \n },\n },\n \"color\" : {\n \"field\" : \"Country\",\n \"type\": \"nominal\",\n }\n }\n \n })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1378,"value":"md`**Table 3.** Explore the underlying data`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1368,"value":"viewof wrangled = Wrangler(siteHistory)","pinned":false,"hidden":false,"mode":"ojs"},{"id":5,"value":"md`### Libraries and data`","pinned":false,"hidden":false,"mode":"ojs"},{"id":9,"value":"d3 = require(\"d3@6\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":1194,"value":"import {select} from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"vegalite = require (\"vega-embed\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"_ = require (\"lodash\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":22,"value":"import {gene_coordinates} from \"@spond/revised-sars-cov-2-analytics-page\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1365,"value":"import {Wrangler, op} from \"@observablehq/data-wrangler\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":692,"value":"site_equiv = _.groupBy (_.toPairs (jsonCodonLookup), (d)=>d[1])","pinned":false,"hidden":false,"mode":"ojs"},{"id":49,"value":"import {jsonCodonLookup} from \"@spond/sars_cov_2_sites\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":58,"value":"import {multiAutoSelect} from \"@spond/untitled/3\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1209,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":1218,"value":"siteList = parseSiteList (_.map (sites, (d)=>d.split (\" \")[1]))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1810,"value":"avtable_plot = _.chain (avtable).map (d=>{\n let r = [];\n _.each (d, (v,c)=>{\n if (c != 'Country') {\n r.push ({'Country' : d.Country, 'Value' : v, 'Gene' : c});\n }\n });\n return r;\n }).flatten ().value()","pinned":false,"hidden":false,"mode":"ojs"},{"id":1439,"value":"all_times = _.toPairs (_.groupBy (_.filter (siteHistory, (d)=>d.alpha < d.beta && d.p < 0.01), (d)=>d.loc))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1550,"value":"all_timesN = _.toPairs (_.groupBy (_.filter (siteHistory, (d)=>d.alpha > d.beta && d.p < 0.01), (d)=>d.loc))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1445,"value":"groupedBySite = _.chain (filteredSH).groupBy((d)=>d.gene + \"/\" + d.site).value()","pinned":false,"hidden":false,"mode":"ojs"},{"id":1697,"value":"commonSites = _.chain (groupedBySite).mapValues((d)=>d.length).toPairs().filter ((d)=>d[1]*2 >= sorted_countries.length).map (d=>d[0]).value()","pinned":false,"hidden":false,"mode":"ojs"},{"id":1402,"value":"pad_array = (arr)=>_.map (arr, (d)=>\"Site \" + d.toUpperCase())","pinned":false,"hidden":false,"mode":"ojs"},{"id":1477,"value":"selectable_sites = _.keys(jsonCodonLookup)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1678,"value":"sorted_countries = _.sortBy ([...countries])","pinned":false,"hidden":false,"mode":"ojs"},{"id":1660,"value":"countries = {\n let not_dates = new Set ();\n _.each (_.countBy (siteHistory, (d)=>d.country), (k,d)=> {\n if (d.split ('-').length == 1) {\n not_dates.add (d);\n }\n })\n return not_dates;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":46,"value":"siteHistory = {\n let raw_data = await d3.csv (\"https://raw.githubusercontent.com/spond/SARS-CoV-2-variation/master/by-country-fel.csv\", d3.autoType);\n //let raw_data = d3.csvParse (await FileAttachment(\"windowed-sites-fel-current@1.txt\").text(), d3.autoType);\n _.each (raw_data, (s)=>{\n s.loc = s.gene.toUpperCase() + \"/\" + s.site;\n /*if (s.loc in siteList[1]) {\n s.label = siteList[1][s.loc]; \n } (else { */\n s.label = s.loc;\n //}\n s.sorter = parseInt(jsonCodonLookup[s.loc]);\n s.p = Math.max (s.p, 1e-16);\n \n });\n return raw_data;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1722,"value":"filteredSH = _.filter (siteHistory, (d)=>countries.has (d.country))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1776,"value":"table1key = {\n let p = table_view_options.indexOf (\"Positive\") >= 0;\n let s = table_view_options.indexOf (\"Scaled\") >= 0;\n if (p) return s ? 'PS' : 'P';\n return s ? 'NS' : 'N';\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1739,"value":"table1data = {\n let table = [];\n _.each (sorted_countries, (country)=> {\n let row = {\n 'Country' : country\n };\n _.each (gene_size, (kb, gene)=> {\n let tag = country + \"/\" + gene;\n row[gene] = {\n 'P' : 0, 'N' : 0, 'PS' : 0, 'NS' : 0\n };\n \n if (tag in countsByGeneCountry) {\n row[gene]['P'] = _.filter (countsByGeneCountry[tag], (d)=>d.alpha < d.beta).length;\n row[gene]['N'] = _.filter (countsByGeneCountry[tag], (d)=>d.alpha > d.beta).length;\n if (row[gene]['P']) {\n row[gene]['PS'] = row[gene]['P'] / kb / countsByGeneCountry[tag][0].T_int;\n }\n if (row[gene]['N']) {\n row[gene]['NS'] = row[gene]['N'] / kb / countsByGeneCountry[tag][0].T_int;\n }\n }\n \n });\n table.push (row)\n });\n return table;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1721,"value":"countsByGeneCountry = _.chain (filteredSH).groupBy ((d)=>d.country + \"/\" + d.gene).value()","pinned":false,"hidden":false,"mode":"ojs"},{"id":1798,"value":"gene_list = _.map (gene_size, (d,k)=>k)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1734,"value":"import {gene_size} from \"@spond/sc2-temporal-selection-trends\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1382,"value":"forPlot = {\n let selected = _.filter (siteHistory, (d)=>siteList[0].has (d.loc) && countries.has (d.country));\n let ss = _.countBy (selected, (d)=>d.loc);\n _.each ([...siteList[0]], (s)=> {\n if (! (s in ss)) {\n selected.push ({\"label\" : s, \"sorter\" : parseInt(jsonCodonLookup[s])})\n }\n });\n return selected;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1259,"value":"nucToCodons = _.chain (_.toPairs (jsonCodonLookup)).groupBy ((d)=>d[1]).mapValues ((d)=>_.map (d, (c)=>c[0])).value()","pinned":false,"hidden":false,"mode":"ojs"},{"id":596,"value":"parseSiteList = (site_list) => {\n let labels = {};\n site_list = _.map (site_list, (d)=>d.split (\"/\"));\n \n let sites = new Set (_.map (site_list, (d)=>{\n if (+d == d) {\n for (let i = d; i > d-3; i--) {\n if (i in nucToCodons) {\n d = nucToCodons[i][0].split (\"/\");\n break;\n }\n }\n \n }\n if (d[0].substr (0,4) == \"ORF1\") {\n let eq = null;\n _.each (site_equiv[jsonCodonLookup [d[0]+\"/\"+d[1]]], (tr)=> {\n if (tr[0].substr (0,4) != \"ORF1\") {\n eq = tr[0];\n labels[eq] = d[0] + \"/\" + d[1];\n }\n });\n return eq;\n }\n return d[0] + \"/\" + d[1];\n}));\n return [sites, labels]\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1341,"value":"params = new URLSearchParams(location.search)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1344,"value":"sitelist = params.get(\"sites\") ? params.get(\"sites\").toUpperCase() : null","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W34","stdlib":"1","titling":"auto"}