31:[] 2f:{"title":"Simple linear regression","cells":[{"id":455,"value":"# Simple linear regression\n## Find parameter estimates, make predictions, and visualize in Observable Plot","pinned":false,"hidden":false,"mode":"md"},{"id":1251,"value":"Simple linear regression is used to describe the relationship between two quantitative variables. The goal of simple linear regression is to find the line of best fit that minimizes the sum of the squared differences between the observed data and the values predicted by the model.\n\nThe interactive chart below shows the _residuals_ for each observation (the difference between the actual and predicted value). The line found by ordinary least squares minimizes the sum of the residuals squared.","pinned":false,"hidden":false,"mode":"md"},{"id":1004,"value":"viewof replay = Inputs.button(\"Change the data!\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":1006,"value":"Plot.plot({\n marks: [\n Plot.ruleX(fullDat, {\n x: \"x\",\n y1: \"predicted\",\n y2: \"y\",\n stroke: \"red\",\n strokeWidth: 2\n }),\n Plot.linearRegressionY(dat, { x: \"x\", y: \"y\", fill: null }),\n Plot.dot(dat, { x: \"x\", y: \"y\", fill: \"black\", r: 6 })\n ],\n width: 500,\n height: 300\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":0,"value":"Here, we walk through simple linear regression (for parameter estimates, R² and _r_ correlation coefficient) using the [SimpleLinearRegression](https://github.com/mljs/regression-simple-linear#readme) function in [ml.js](https://github.com/mljs), a JavaScript library providing methods for linear and logistic regression (along with machine learning, optimization, and linear algebra methods). Then we add model outputs to an Observable Plot chart.","pinned":false,"hidden":false,"mode":"md"},{"id":805,"value":"note(htl.html`Just want to visualize simple linear regression in Observable Plot? The linearRegression mark \"draws linear regression lines with confidence bands, representing the estimated linear relation of a dependent variable (typically y) on an independent variable (typically x).\"`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":14,"value":"---\n## Getting started","pinned":false,"hidden":false,"mode":"md"},{"id":886,"value":"### Loading the `ml.js` library","pinned":false,"hidden":false,"mode":"md"},{"id":17,"value":"Use [require](https://observablehq.com/@observablehq/require) to load the `ml.js` library into your Observable notebook (here, as `ML`):","pinned":false,"hidden":false,"mode":"md"},{"id":8,"value":"ML = require(\"https://www.lactame.com/lib/ml/6.0.0/ml.min.js\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":461,"value":"---\n## Prepping your data\n","pinned":false,"hidden":false,"mode":"md"},{"id":27,"value":"For this example, we will explore the relationship between penguin body mass and flipper length using the [built-in](https://github.com/observablehq/stdlib/blob/77e77c5/src/library.js#L60) [_penguins_](https://allisonhorst.github.io/palmerpenguins/) dataset (shown in the scatterplot below).","pinned":false,"hidden":false,"mode":"md"},{"id":882,"value":"// The penguins dataset is automatically available in all Observable notebooks.\npenguins","pinned":true,"hidden":false,"mode":"ojs"},{"id":469,"value":"// Observable Plot is part of Observable's Standard Library, so can be automatically called in any Observable notebook\nPlot.plot({\n marks: [\n Plot.dot(penguins, {x: \"body_mass_g\", y: \"flipper_length_mm\"})\n ]\n})","pinned":true,"hidden":false,"mode":"ojs"},{"id":494,"value":"### Omit missing values (`NaN` and `null`) from model variables","pinned":false,"hidden":false,"mode":"md"},{"id":499,"value":"The function we’ll use for regression in ml.js requires that all observations (rows) have numeric values for both body mass and flipper length. This is because unlike other JavaScript methods (e.g. d3-array methods) that are robust with respect to null and NA values, the SimpleLinearRegression function in ml.js does not exclude them on its own: any `NaN` values will lead to `NaN`s returned for all model estimates, and any **null** values are treated as zeroes. \n","pinned":false,"hidden":false,"mode":"md"},{"id":990,"value":"note(htl.html`Note: Be sure to check for NaN and null values in your variables, as they can lead to unexpected (or incorrect!) model estimates. In particular, beware that null values are NOT omitted but are instead treated as zeroes, so will change model estimates.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":987,"value":"Below, we confirm that there are two penguins where either *flipper_length_mm* or *body_mass_g* (or in this case, both) are `NaN`:","pinned":false,"hidden":false,"mode":"md"},{"id":879,"value":"penguins.filter(d => isNaN(d.flipper_length_mm) || isNaN(d.body_mass_g))","pinned":true,"hidden":false,"mode":"ojs"},{"id":877,"value":"Below, we create a subset of the penguins data omitting any observations where flipper length or body mass is `NaN`. This step is unnecessary if your variables of interest don’t have NaN or null values. ","pinned":false,"hidden":false,"mode":"md"},{"id":477,"value":"penguinsComplete = penguins.filter(d => !isNaN(d.flipper_length_mm) && !isNaN(d.body_mass_g))","pinned":true,"hidden":false,"mode":"ojs"},{"id":504,"value":"## Simple linear regression by ordinary least squares","pinned":false,"hidden":false,"mode":"md"},{"id":747,"value":"We can perform simple linear regression by ordinary least squares using the `SimpleLinearRegression` function from ml.js with the following syntax: \n\n>>`ML.SimpleLinearRegression(independentVariable, dependentVariable)`","pinned":false,"hidden":false,"mode":"md"},{"id":864,"value":"We input penguin flipper length and body mass and the independent and dependent variables, respectively, assigning the output as `penguins_lm` (short for “penguins linear model”).","pinned":false,"hidden":false,"mode":"md"},{"id":474,"value":"penguins_lm = new ML.SimpleLinearRegression(penguinsComplete.map(d => d.flipper_length_mm), penguinsComplete.map(d => d.body_mass_g))","pinned":true,"hidden":false,"mode":"ojs"},{"id":522,"value":"### Access model coefficients","pinned":false,"hidden":false,"mode":"md"},{"id":875,"value":"Simple linear regression models the relationship between a dependent variable and an independent variable using a straight line, defined by coefficients for the intercept and slope. We can access model coefficients as follows (here, storing them as an array named `penguins_coefficients`).","pinned":false,"hidden":false,"mode":"md"},{"id":529,"value":"penguins_coefficients = penguins_lm.coefficients // [intercept, slope]","pinned":true,"hidden":false,"mode":"ojs"},{"id":566,"value":"Here are the individual coefficients:","pinned":false,"hidden":false,"mode":"md"},{"id":568,"value":"penguins_coefficients[0] // Intercept estimate","pinned":true,"hidden":false,"mode":"ojs"},{"id":571,"value":"penguins_coefficients[1] // Slope estimate","pinned":true,"hidden":false,"mode":"ojs"},{"id":535,"value":"### Get Pearson’s _r_, _R_², and root-mean-square deviation (RMSD)","pinned":false,"hidden":false,"mode":"md"},{"id":873,"value":"ml.js `SimpleLinearRegression` also offers [additional methods](https://github.com/mljs/regression-simple-linear/blob/master/src/index.js) including the score function shown below, which returns Pearson’s _r_ (correlation coefficient), _R²_ and root-mean-square deviation for the model.","pinned":false,"hidden":false,"mode":"md"},{"id":540,"value":"penguins_corr = penguins_lm.score(penguinsComplete.map(d => d.flipper_length_mm), penguinsComplete.map(d => d.body_mass_g))","pinned":true,"hidden":false,"mode":"ojs"},{"id":545,"value":"As above, we can access individual pieces from the score output:","pinned":false,"hidden":false,"mode":"md"},{"id":547,"value":"penguins_corr.r // Pearson's r","pinned":true,"hidden":false,"mode":"ojs"},{"id":549,"value":"penguins_corr.r2 // R^2 coefficient of determination","pinned":true,"hidden":false,"mode":"ojs"},{"id":559,"value":"penguins_corr.rmsd // Root-mean-square deviation","pinned":true,"hidden":false,"mode":"ojs"},{"id":600,"value":"---\n## Estimate new values using the model","pinned":false,"hidden":false,"mode":"md"},{"id":709,"value":"Next, we'll use `.predict` and .`computeX` to predict values of the y- and x-variable, respectively, based on the model estimates.","pinned":false,"hidden":false,"mode":"md"},{"id":602,"value":"Predict _y_: Estimate the mean body mass of penguins with a 205 mm flipper length.","pinned":false,"hidden":false,"mode":"md"},{"id":604,"value":"penguins_lm.predict(205)","pinned":true,"hidden":false,"mode":"ojs"},{"id":606,"value":"Predict _x_: Estimate the mean flipper length of 5000 g penguins.","pinned":false,"hidden":false,"mode":"md"},{"id":609,"value":"penguins_lm.computeX(5000)","pinned":true,"hidden":false,"mode":"ojs"},{"id":577,"value":"---\n## Add model information to an Observable Plot chart","pinned":false,"hidden":false,"mode":"md"},{"id":749,"value":"You may want to visualize your data, model, and model estimates in one place. Below, we add model estimates (coefficients, *R²* and Pearson’s _r_) to a scatterplot chart showing the linear relationship for penguin flipper length and body mass.","pinned":false,"hidden":false,"mode":"md"},{"id":778,"value":"The model trendline (in black) and confidence band (in gray) are calculated by Observable Plot's [linearRegression mark](https://observablehq.com/plot/marks/linear-regression). The text annotation for model estimates (in blue) are pulled from the model outputs computed by ml.js above.","pinned":false,"hidden":false,"mode":"md"},{"id":579,"value":"chart = Plot.plot({\n marks: [\n Plot.dot(penguinsComplete, { // Add scatterplot (blue points)\n x: \"flipper_length_mm\",\n y: \"body_mass_g\",\n fill: \"blue\",\n r: 4,\n opacity: 0.4\n }),\n Plot.linearRegressionY(penguinsComplete, { // Adds regression line and confidence interval\n x: \"flipper_length_mm\",\n y: \"body_mass_g\"\n }),\n Plot.text( // Add text annotation for model estimates (shown in blue)\n [`y = ${penguins_coefficients[1].toFixed(1)}x - ${-penguins_coefficients[0].toFixed(1)}\n \n 𝘙 ² = ${penguins_corr.r2.toFixed(2)}\n \n 𝘳 = ${penguins_corr.r.toFixed(2)}` \n ],\n {frameAnchor: \"bottom-right\", \n dy: -20,\n fontSize: 16, \n fill: \"blue\", \n fontWeight: 600} \n )\n ]\n})","pinned":true,"hidden":false,"mode":"ojs"},{"id":738,"value":"### Why ml.js? Are there other options?","pinned":false,"hidden":false,"mode":"md"},{"id":740,"value":"Absolutely! A number of helpful JavaScript libraries have functions to compute estimates for simple linear regression, including [Simple Statistics](http://simple-statistics.github.io/), [regression-js](http://tom-alexander.github.io/regression-js/), [ml.js](https://github.com/mljs/ml), and Observable Plot (to draw the model and confidence interval). We chose to use `ml.js` here because the library also supports additional regression and machine learning methods, so we hope that learning simple linear regression in ml.js can be a gateway to discovering other useful tools for regression and ML in the same library. ","pinned":false,"hidden":false,"mode":"md"},{"id":1246,"value":"## Appendix","pinned":false,"hidden":false,"mode":"md"},{"id":803,"value":"import {note} from \"@observablehq/notes\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1005,"value":"x = {\n replay;\n const temp = [1, 3, 4, 4.5, 9, 11, 11.2, 13, 14.5];\n return temp;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1007,"value":"y = x.map((d) => d * 2.5 + 50 * Math.random() * Math.random())","pinned":false,"hidden":false,"mode":"ojs"},{"id":1008,"value":"dat = x.map((d, i) => ({ x: d, y: y[i] }))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1010,"value":"// Linear model:\ndatLM = new ML.SimpleLinearRegression(\n dat.map((d) => d.x),\n dat.map((d) => d.y)\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1011,"value":"// Predictions at all x values from model:\nlmPredict = x.map((d) => datLM.predict(d))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1012,"value":"fullDat = dat.map((d, i) => ({ ...d, predicted: lmPredict[i] }))","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}