31:T99b,md`As algorithmic accountability garners coverage in the news, it also grows as a research field among academia and becomes essential to computer-science and data-science education. This semester, at New York University, I had the fortune to study under Julia Stoyanovich, who is part of a movement called [Data Responsibly](https://dataresponsibly.github.io/). Their credo: to "study the foundations of responsible data science, and build tools that translate our insights into data science practice." One of these tools, RankingFacts, uses data visualization to help humans interpret how machine-assisted decision tools generate outcomes. The outcomes are in terms of rank, conveying who is more favorable and who is less in the "eyes" of the system (so that, for example, they might or might not be qualified for a loan). The visualization ensemble, i.e., dashboard, is theorized in this [paper](http://sites.computer.org/debull/A19sept/p13.pdf) from Stoyanovich and Bill Howe (University of Washington), and is analogous to nutrition labels in food products, where "simple, standard labels convey information about the ingredients and production processes." In a [web demo version](http://demo.dataresponsibly.com/rankingfacts/), it can also be generated on any CSV file to help users better understand how, and to what extent, specific attributes can affect the stability, diversity, and fairness of the ranks given to individuals of a group (for example, a group based on race or sex). The demo version is currently limited to rank scores as *linear combinations* of attribute values, and uses Plotly to automate hand-designed tables and charts. However, the paper does propose automating the designing of charts using [Draco](https://uwdata.github.io/draco/), a constraint-based framework that recommends visualization designs based on input datasets. The result is a "nutrition label" dashboard whose layout adapts to different datasets to provide complete information and avoid human design bias. This notebook is an experiment in using Draco to fully automate the creation of said charts. Datasets are provided via URLs (more details below) and read with [Danfo.js](https://danfo.jsdata.org/). The reason I chose Danfo is that its DataFrames have convenient wrangling methods and can be converted into Tensors for fast calculations. Finally, this will likely be a work in progress for a while, so please fork and make changes as you see fit!`32:T481,// Danfo.js only accepts CSVs from the internet and has no methods for parsing local CSVs into DataFrames. If anyone knows a workaround, please comment! // Right now, there are some hoops to jump if you want to upload a local file: // (1) Fork this notebook and upload your file in the fork. // (2) Publish the fork so that the file lives in Observable's server (ergo the internet) as opposed to just a blob URL that danfo won't accept. Now it has an URL that can be fed into danfo.read_csv(). // (3) Unpubish the fork and read in the file using the new, non-blob URL. // Default URL: COMPAS dataset acquired by ProPublica. Refer to Northpointe-ProPublica controversy: // * News article: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing // * Northpointe's response: https://www.equivant.com/response-to-propublica-demonstrating-accuracy-equity-and-predictive-parity/ // * ProPublica's rejoinder: https://www.propublica.org/article/propublica-responds-to-companys-critique-of-machine-bias-story url = "https://raw.githubusercontent.com/DataResponsibly/RankingFacts/master/NotebookVersion/dataset/COMPAS_gender.csv"33:T5b8,// https://next.observablehq.com/@tvirot/non-reactive-control-flow generate = { const button = html``; button.disabled = updated; button.onclick = async () => { // Fairness const fairness = [...fairnessCheckbox.querySelectorAll("input")] .filter(x => x.checked) .map(x => fairnessCheckbox.options[+x.value]); // Diversity const diversity = [...diversityCheckbox.querySelectorAll("input")] .filter(x => x.checked) .map(x => diversityCheckbox.options[+x.value]); if (fairness.length === 0) { mutable error = new Error( "Must choose at least one attribute for fairness." ); return; } if (diversity.length === 0) { mutable error = new Error( "Must choose at least one attribute for diversity." ); return; } mutable error = null; // Ranking function const vars = viewof rankingSliders.options; const coefs = [...viewof rankingSliders.html.querySelectorAll("form")].map( form => form.querySelector("input[type=range]").valueAsNumber ); // Release tensors for garbage collector if (params.X) params.X.dispose(); if (params.y) params.y.dispose(); const X = df_processed.loc({ columns: vars }).tensor; const y = X.dot(coefs); mutable params = { fairness, diversity, vars, coefs, X, y }; mutable updated = true; }; return button; }34:T420,md`___ ## Appendix There are too many components at work for which I'd like to thank, and that's the beauty of Observable ;) - Nikhil Thorat for the [introduction](https://next.observablehq.com/@nsthorat/introduction-to-deeplearn-js) to TensorFlow on Observable. - "Ta" Virot C. for [how to surpress Observable reactivity](https://next.observablehq.com/@tvirot/non-reactive-control-flow) so that the notebook runs much more efficiently (although this low-key feels illegal). - Visnu Pitiyanuvath for the [introduction](https://next.observablehq.com/@visnup/hello-danfo-js) to Danfo.js. - Dominik Moritz for the [introduction](https://observablehq.com/@domoritz/hello-draco?ui=classic) to Draco, and Benoît Cantagrel for [how to use Draco on a non-tutorial dataset](https://observablehq.com/@bcantagrel/draco?ui=classic). - Jeffrey Heer for the speedy [introduction](https://next.observablehq.com/@uwdata/introduction-to-vega-lite) to Vega-Lite. - And Mike Bostock, Philippe Rivière, and the Observable team for the gold standard of Observable inputs!`35:[] 2f:{"title":"Nutritional Labels for Datasets with Draco","cells":[{"id":0,"value":"md`# Nutritional Labels for Datasets with Draco`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1326,"value":"md`\n

Image: [K-State Research and Extension](https://www.ksre.k-state.edu/news/images/news-stories/2020/family-con-science/nutrition-label-new.jpg)