2f:T48b,Throughout history, 'tools' have been invented/devloped to assist our [cognition](https://en.wikipedia.org/wiki/Cognition) with multi-dimensional/variable systems: * [Euclidean Geometry](https://en.wikipedia.org/wiki/Euclidean_geometry) * [Calculus](https://en.wikipedia.org/wiki/Calculus) * [Linear Systems](https://en.wikipedia.org/wiki/Linear_system) * [Hyperbolic geometry](https://en.wikipedia.org/wiki/Hyperbolic_geometry) * [Imaginary Number](https://en.wikipedia.org/wiki/Imaginary_number) * [Economics](https://en.wikipedia.org/wiki/Economics) * [Legal Systems](https://en.wikipedia.org/wiki/List_of_national_legal_systems) * [Philosophy](https://en.wikipedia.org/wiki/Philosophy) * [Matrix Mathematics](https://en.wikipedia.org/wiki/Matrix_(mathematics%29) * [Perspective](https://observablehq.com/@mariodelgadosr/perspective?collection=@mariodelgadosr/observable-for-active-retirement-communities#cell-446) > **Question**: Is this cube a three-dimensional object?: > ${canvas} > --[Projection of 3D Models using JavaScript and HTML5 Canvas](https://observablehq.com/d/6e3bcbbcfee2f7b6) * [Tensors](https://en.wikipedia.org/wiki/Tensor)30:T577,## [Logistic Model](https://en.wikipedia.org/wiki/Logistic_regression) A logistic model statistically models the probability of some binary outcome by assuming that the [log-odds](https://en.wikipedia.org/wiki/Logit) of the outcome is a linear combination of its predictors. See [***Just Enough Statistics/Logistic Regression***](https://observablehq.com/d/1f44a61d095d98d4?collection=@mariodelgadosr/youre-never-too-old-to-code#cell-399) for an interactive example. The student pass/fail example in [the Wikipedia Logistic regression article](https://en.wikipedia.org/wiki/Logistic_regression#Example) is used to illustrate the building of a logistic model with TensorFlow.js. As [previously demonstrated](https://observablehq.com/d/1f44a61d095d98d4?collection=@mariodelgadosr/youre-never-too-old-to-code#cell-399), a logistic model is based on the following relationship: ${tex.options({ displayMode: true })`Pr(y) = \frac{1}{1+e^{-(\beta_0 + \beta_1\cdot x)}}`} ${tex`Pr(y)`} (the prediction that a student will either pass or fail an exam as a function of how much they studied) is a [sigmoid function](https://en.wikipedia.org/wiki/Sigmoid_function) in the following format: ${tex.options({ displayMode: true })`sigmoid(z) = \frac{1}{1+e^{-z}}`} In logistic models, the term ${tex`\beta_0 + \beta_1\cdot x`} is also referred to as a [logit](https://en.wikipedia.org/wiki/Logit). 31:T5e8,{ const x1 = d3 .range(...[0, 6], 0.1) // https://github.com/d3/d3-array/blob/v3.2.3/README.md#range .concat(6); // https://github.com/d3/d3-array/issues/90 const yProbility = x1.map((x) => 1 / (1 + Math.pow(Math.E, -(B0 + B1 * x)))); const w = width * 0.6; const h = w * 0.5; return Plot.plot({ width: w, height: h, x: { domain: [0, 6], label: "x", grid: true }, y: { domain: [0, 1], label: "y", grid: true }, marks: [ Plot.ruleX([0]), // y axis Plot.ruleY([0]), // x axis Plot.line(x1, { x: x1, y: yProbility }), // y = P(x) showLocationParameter ? Plot.dot([[-B0 / B1, 0.5]], { fill: "red", r: 5, title: (d) => `x: ${d3.format(".3")(d[0])}, y: ${d[1]}` }) : [], // location parameter https://en.wikipedia.org/wiki/Location_parameter showLocationParameter ? Plot.line( [ [0, 0.5], [-B0 / B1, 0.5] ], { stroke: "blue", strokeDasharray: [5, 5] } ) : [], showLocationParameter ? Plot.line( [ [-B0 / B1, 0], [-B0 / B1, 0.5] ], { stroke: "blue", strokeDasharray: [5, 5] } ) : [], () => svg` ${MathJax.tex2svg( String.raw`Pr(y) = \frac{1}{1+e^{-(${B0} + ${B1}\cdot x)}}` ).querySelector("svg")} ` ] }); }32:T625,### [Logit](https://en.wikipedia.org/wiki/Logit) Proof
If ${tex.block` p = \frac{1}{1+e^{-(\beta_0 + \beta_1\cdot x)}}`} And ${tex.options({ trust: true, displayMode: true })`odds = \left(\frac{p}{\href{https://en.wikipedia.org/wiki/Bernoulli_distribution}{1-p}} \right) `} Then ${tex.options({ trust: true, displayMode: true })`\begin{aligned} \href{https://en.wikipedia.org/wiki/Logit}{logit\text{ } odds} &= \ln\left(\frac{p}{1-p} \right) \\ &= \beta_0 + \beta_1\cdot x \end{aligned}`} Proof: ${tex.block` \begin{aligned} Let \space {z} &= (\beta_0 + \beta_1\cdot x) \\ p &= \frac{1}{1+e^{-z}} \\ odds &= \frac{p}{1-p} \\ &= \frac{\frac{1}{1+e^{-z}}}{1-\frac{1}{1+e^{-z}}} \\ &= \frac{\frac{1}{1+e^{-z}}}{1-\frac{1}{1+e^{-z}}} \cdot \frac{1+e^{-z}}{1+e^{-z}} \\ &= \frac{\frac{1+e^{-z}}{1+e^{-z}}}{1+e^{-z}-\frac{1+e^{-z}}{1+e^{-z}}} \\ &= \frac{1}{1+e^{-z}-1} \\ &= \frac{1}{e^{-z}} \\ &= e^{{z}} \\ &= e^{(\beta_0 + \beta_1\cdot x)} \\ logit\text{ } odds &= \ln\left(\frac{p}{1-p} \right) \\ &= \ln\left(e^{(\beta_0 + \beta_1\cdot x)}\right) \\ &= \beta_0 + \beta_1\cdot x \end{aligned} `} [${tex`Q.E.D`}](https://en.wikipedia.org/wiki/Q.E.D.) Related: Abraham Lincoln's 'love' of the proof: [Lincoln, Euclid, and the Satisfaction of Success](https://quod.lib.umich.edu/j/jala/2629860.0041.104/--lincoln-euclid-and-the-satisfaction-of-success?rgn=main;view=fulltext)33:T460,{ const plotlyData = { x: [perfectRetirement[0]], y: [perfectRetirement[1]], z: [perfectRetirement[2]], mode: "markers", marker: { size: 8, color: "red" }, type: "scatter3d" }; const layout = { height: 400, margin: { autoexpand: false, l: 0, r: 0, b: 0, t: 0, pad: 0 }, scene: { camera: { eye: { x: -2.25, y: -2.25, z: 1 //1.25 } }, xaxis: { anchor: "free", position: 0, title: { text: "Affordability" }, range: [0, 1], nticks: 10, tickformat: ".0%" }, yaxis: { title: { text: "Costco" }, range: [0, 1], nticks: 10, tickformat: ".0%" }, zaxis: { title: { text: "Weather" }, range: [0, 1], nticks: 10, tickformat: ".0%" } } }; // https://plotly.com/javascript/plotlyjs-function-reference/#plotlyreact Plotly.react(plotlyDiv, [plotlyData], layout); return md`⏎ Click cell to see Plotly 3D Plot code`; }34:[{"id":"a20dec6cf7861a12c29b78f0f004e43cda3189bb69431f1f73b6dc4f007753e34b56460e40d2f499c7e063f925597f4c2133f4b9abaf4afb7462eeb3e259fd8b","href":"https://static.observableusercontent.com/files/a20dec6cf7861a12c29b78f0f004e43cda3189bb69431f1f73b6dc4f007753e34b56460e40d2f499c7e063f925597f4c2133f4b9abaf4afb7462eeb3e259fd8b","name":"image.png","type":"image/png","lastModified":1684693673390.153,"size":55961}] 2d:{"title":"A Quick Introduction to Tensors","cells":[{"id":807,"value":"# A Quick Introduction to Tensors\n\nFor a detailed discussion on TensorFlow.js see [@jeyabbalas](https://observablehq.com/@jeyabbalas)' [TensorFlow.js collection](https://observablehq.com/collection/@jeyabbalas/tensorflow-js).","pinned":false,"hidden":false,"mode":"md"},{"id":855,"value":"toc(\"h2,h3,h4, h5\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":813,"value":"## Thinking Multi-dimensionally\nAs humans, we are optimized for a lot of multi-dimensional tasks; like:\n* Stereoscopic [vision](https://amplifyeyecareolympia.com/vision-therapy/the-importance-of-stereoscopic-vision/) and [hearing](https://www.asianscientist.com/2015/05/in-the-lab/hearing-stereo-brain-balances-left/)\n* [Walking](https://press.uchicago.edu/ucp/books/book/chicago/S/bo22782753.html)\n* [Decision Making](https://www.annualreviews.org/doi/abs/10.1146/annurev-psych-010814-015310)","pinned":false,"hidden":false,"mode":"md"},{"id":824,"value":"### Model a Perfect Retirement Location with 3 Variables \n\n* ${pc(perfectRetirement[0])} Affordable;\n* ${pc(perfectRetirement[2])} Great weather;\n* ${pc(perfectRetirement[1])} Near a Costco.","pinned":false,"hidden":false,"mode":"md"},{"id":821,"value":"viewof perfectRetirement = ternarySlider({\n value: [0.5, 0.25, 0.25],\n labels: [\"Affordability\", \"Costco\", \"Weather\"]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":995,"value":"### Tools\nHumans (and [other species](https://www.livescience.com/9761-10-animals-tools.html)) are 'pre-wired' for using 'tools':","pinned":false,"hidden":false,"mode":"md"},{"id":1668,"value":"[The Human Spark | Learning Right and Wrong ${await FileAttachment(\"image.png\").image()}](https://florida.pbslearningmedia.org/resource/hs11.living.gen.birth.learnright/learning-right-and-wrong/)\n\nAnd we're naturally altruistic [and selfish](https://en.wikipedia.org/wiki/The_Selfish_Gene):\n\n","pinned":false,"hidden":false,"mode":"md"},{"id":844,"value":"$2f","pinned":false,"hidden":false,"mode":"md"},{"id":0,"value":"## Tensors and TensorFlow\n\nIn [machine learning](https://en.wikipedia.org/wiki/Machine_learning), [*Tensors*](https://en.wikipedia.org/wiki/Tensor_(machine_learning%29) are a tool/strategy to organize, represent and transform data. \n\n \n\n* They are fundamental data structures in [*Tensor*Flow](https://en.wikipedia.org/wiki/TensorFlow) and are therefore immortalized in the first part of its name. Tensors are simply high-dimensional matrices. The *Flow* part of Tensor*Flow* alludes to the operations that can be performed on tensors. \n\n* [TensorFlow.js](https://www.tensorflow.org/js/) is a JavaScript library that enables TensorFlow for browsers.\n\n","pinned":false,"hidden":false,"mode":"md"},{"id":979,"value":"### Example: Create and Square a Tensor\nHere's an example of applying a squaring operation to a matrix:\n\n* ${tex` \\text{Matrix Notation: }\\begin{bmatrix} 1 & 2 & 3\\end{bmatrix}^2 \\rightarrow \\begin{bmatrix}1 & 4 & 9\\end{bmatrix}`}\n\n* ${tex.options({\n trust: true,\n})` \\text{TensorJS syntax: }tf.tensor[1, 2, 3].\\href{https://js.tensorflow.org/api/latest/?_gl=1*17iifso*_ga*MTY3NTk5MTMzMy4xNjgxNDg3NDQ2*_ga_W0YLR4190T*MTY4MjUyMzMyOC43LjEuMTY4MjUyMzQwNC4wLjAuMA..#pow}{pow}(2)`} \n\n* ${tex` \\text{Resultant Tensor: } \\begin{bmatrix} 1,4,9\\end{bmatrix}`} \n\n\n[Operators](https://js.tensorflow.org/api/latest/?_gl=1*17iifso*_ga*MTY3NTk5MTMzMy4xNjgxNDg3NDQ2*_ga_W0YLR4190T*MTY4MjUyMzMyOC43LjEuMTY4MjUyMzQwNC4wLjAuMA..#Operations), like [pow](https://js.tensorflow.org/api/latest/?_gl=1*17iifso*_ga*MTY3NTk5MTMzMy4xNjgxNDg3NDQ2*_ga_W0YLR4190T*MTY4MjUyMzMyOC43LjEuMTY4MjUyMzQwNC4wLjAuMA..#pow), take tensors as input and give tensors as outputs. \n","pinned":false,"hidden":false,"mode":"md"},{"id":970,"value":"squaredTensor = tf.tensor([1, 2, 3]).pow(2)","pinned":true,"hidden":false,"mode":"ojs"},{"id":981,"value":"### Retrieve the Data Results of an Operation on the Tensor","pinned":false,"hidden":false,"mode":"md"},{"id":977,"value":"squaredTensor.data()","pinned":true,"hidden":false,"mode":"ojs"},{"id":984,"value":"### [Format Display](https://observablehq.com/@tmcw/pt) of the Resultant Tensor","pinned":false,"hidden":false,"mode":"md"},{"id":104,"value":"pt(squaredTensor)","pinned":true,"hidden":false,"mode":"ojs"},{"id":196,"value":"$30","pinned":false,"hidden":false,"mode":"md"},{"id":560,"value":"### Calculating the Logits\n\nA [logit](https://en.wikipedia.org/wiki/Logit) is \na convenient linear expression that is derived from the following relationship; the log-odds of the outcome:\n\nIf \n\n${tex.options({\n displayMode: true\n})`Pr(y) = \\frac{1}{1+e^{-(\\beta_0 + \\beta_1\\cdot x)}}`} \n\nAnd \n\n${tex.options({\n displayMode: true\n})`odds = \\frac{Pr(y)}{1-Pr(y)}`} \n\n\nThen \n\n${tex.options({\n trust: true,\n displayMode: true\n})`\\href{https://en.wikipedia.org/wiki/Logit}{logit\\text{ } odds}= \\ln\\left(\\frac{Pr(y)}{1-Pr(y)} \\right) \\\\ = \\beta_0 + \\beta_1\\cdot x `}\n\nA [mathematical proof](https://observablehq.com/d/f29b4d8b1a4acf9b?collection=@mariodelgadosr/youre-never-too-old-to-code#cell-1169) is available below showing its derivation. ","pinned":false,"hidden":false,"mode":"md"},{"id":1009,"value":"#### Data\nThe model will take as input a linear combination of:\n\n* The [independent](https://en.wikipedia.org/wiki/Dependent_and_independent_variables) *predictor* variable of number of hours studied;\n* The [dependent](https://en.wikipedia.org/wiki/Dependent_and_independent_variables) *outcome* Pass/Fail results in binary (0,1) .\n\nThe data is provided as follows: ","pinned":false,"hidden":false,"mode":"md"},{"id":375,"value":"dataNumHoursStudiedPredictor = [0.50, 0.75, 1.00, 1.25, 1.50, 1.75, 1.75, 2.00, 2.25, 2.50, \n 2.75, 3.00, 3.25, 3.50, 4.00, 4.25, 4.50, 4.75, 5.00, 5.50]","pinned":false,"hidden":false,"mode":"ojs"},{"id":388,"value":"dataStudentPassedExamOutcome = [0, 0, 0, 0, 0, 0, 1, 0, 1, 0, \n 1, 0, 1, 0, 1, 1, 1, 1, 1, 1]","pinned":false,"hidden":false,"mode":"ojs"},{"id":1063,"value":"Inputs.table(\n dataNumHoursStudiedPredictor.map((h, i) => ({\n \"Number of Hours Studied\": h,\n \"Pass/Fail\": dataStudentPassedExamOutcome[i],\n Notes: !dataStudentPassedExamOutcome[i] ? \"Failed\" : \"Passed\"\n })),\n { width: 700 }\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1560,"value":"#### Estimating the Parameters with TensorFlow.js\nCoefficients ${tex`\\beta_0`}and ${tex`\\beta_1`} *shape* the function's curve and are numerically dependent on the nature of the input data. \n\nIn the [Wikipedia example](https://en.wikipedia.org/wiki/Logistic_regression#Example) there's a discussion on how [estimates](https://en.wikipedia.org/wiki/Logistic_regression#Parameter_estimation) are determined with [numerical methods](https://en.wikipedia.org/wiki/Logistic_regression#Parameter_estimation).\n\n\nThe [Estimating Logistic Regression Parameters](https://observablehq.com/d/907eea004b9a540a?collection=@mariodelgadosr/youre-never-too-old-to-code) notebook illustrates the power of TensorFlow.js to do the numerical processing for the estimation of the coefficients:","pinned":false,"hidden":false,"mode":"md"},{"id":1643,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":1656,"value":"As demonstrated, TensorFlow.js can be used to determine the regression's parameters by implementing a [optimizaztion](https://www.tensorflow.org/js/guide/train_models#optimizer_loss_and_metric) of a [cross entropy](https://en.wikipedia.org/wiki/Cross_entropy) analysis. \n\nThere values are:\n${tex.options({\n displayMode: true\n})`\\beta_0 \\approx ${estimates.B0}`}\n${tex.options({\n displayMode: true\n})`\\beta_1 \\approx ${estimates.B1}`}","pinned":false,"hidden":false,"mode":"md"},{"id":1478,"value":"estimates = ({ B0: -4.1, B1: 1.5 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1483,"value":"### Visualizing the Effects of the Beta Coefficients on the Sigmoid Function\n\nTo illustrate how the models estimates on ${tex`\\beta_0`} and ${tex`\\beta_1`} affect the predictions, you can adjust the coefficients below to see how they alter the predictive nature of the model. \n\nThe [location paramater](https://en.wikipedia.org/wiki/Location_parameter) (LP) is the midpoint of the curve where \n\n${tex.block`\n\\begin{aligned}\nPr(y) &= \\frac{1}{2} \\space for \\\\ \n LP_x &\\approx \\frac{-B_0}{B_1} \\\\ \n &\\approx \\frac{${-B0}}{${B1}} \\\\ \n &\\approx ${B1 != 0 ? d3.format(\".3\")(-B0/B1) : `-∞`} \n\\end{aligned}\n`}\n ","pinned":false,"hidden":false,"mode":"md"},{"id":1011,"value":"viewof B0 = Inputs.range([-10, 0], {\n step: 0.1,\n label: tex`\\LARGE \\beta_0`,\n value: estimates.B0\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1015,"value":"viewof B1 = Inputs.range([0, 4], {\n step: 0.1,\n label: tex`\\LARGE \\beta_1`,\n value: estimates.B1\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1349,"value":"viewof showLocationParameter = Inputs.toggle({\n label: \"Show Location Parameter\",\n value: true,\n width: 300\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1019,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":476,"value":"#### The Predictions Function Derived with the Estimated Paramaters\nWhile Tensorflow.js could be utilized to implement the prediction function (see syntax in the comments) the heavy lifting in estimating the ${tex`B_0`} and ${tex`B_1`} parameters by TensorFlow.js are now utilized with a relatively simple JavaScript function used to implement the prediction logic that calculates the Pass/Fail probabilities:","pinned":false,"hidden":false,"mode":"md"},{"id":1543,"value":"/* Using Tensor.js \n predict = (x) =>\n tf.sigmoid(\n tf\n .scalar(estimates.B0)\n .add(tf.scalar(estimates.B1).mul(tf.tensor1d([x])))\n ) */\npredict = (x) => 1 / (1 + Math.pow(Math.E, -(estimates.B0 + estimates.B1 * x)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":677,"value":"#### Plotting the predictions\n\nWith the derivation of the prediction function, a plot of the number of hours studied by the probability of passing the exam can be generated.\n\nNote that the predictions are based on the logistic regression and are just that; predictions. \n\nThe table below shows:\n\n* Some students that had a low amount of study hours, passed with a low probability of passing;\n* Some students with high study hours were predicted to pass, but failed.","pinned":false,"hidden":false,"mode":"md"},{"id":1269,"value":"Inputs.table(\n dataNumHoursStudiedPredictor.map((h, i) => ({\n \"Number of Hours Studied\": h,\n \"Pass/Fail\": dataStudentPassedExamOutcome[i],\n Notes: !dataStudentPassedExamOutcome[i] ? \"Failed\" : \"Passed\",\n \"Prediction from Model\": predict(h)\n })),\n { width: 800 }\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":679,"value":"{\n let trace = {\n x: dataNumHoursStudiedPredictor,\n y: dataNumHoursStudiedPredictor.map((d) => predict(d)),\n mode: \"markers\",\n type: \"scatter\"\n };\n\n let data = [trace];\n\n let layout = {\n title: \"Predicted probability of passing according to the logistic model\",\n xaxis: {\n title: {\n text: \"Number of hours studied\"\n }\n },\n yaxis: {\n title: {\n text: \"Probability of passing the exam\"\n },\n range: [0, 1]\n }\n };\n\n const div = htl.html`

`;\n Plotly.newPlot(div, data, layout);\n\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":682,"value":"We can make inferences from this plot, for example, if you want to have at least a 50% chance of passing the exam, you must study for at least 2.75 hours.","pinned":false,"hidden":false,"mode":"md"},{"id":198,"value":"## Appendix","pinned":false,"hidden":false,"mode":"md"},{"id":1169,"value":"$32","pinned":false,"hidden":false,"mode":"md"},{"id":1285,"value":"### Notebook Objects","pinned":false,"hidden":false,"mode":"md"},{"id":1691,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":1738,"value":"{\n const tSVG = viewof perfectRetirement;\n\n tSVG.insertAdjacentElement(\"afterend\", plotlyDiv);\n\n return md`Insert Plotly Plot along side Ternary SVG Element \n⏎ Click cell to see code`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1735,"value":"// Customization: https://observablehq.com/@yurivish/ternary-slider#cell-439\nplotlyDiv = {\n const svgData = viewof perfectRetirement;\n const div = htl.html`

`;\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":816,"value":"import { ternarySlider } from \"@yurivish/ternary-slider\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":671,"value":"Plotly = require(\"https://cdn.plot.ly/plotly-2.14.0.min.js\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":6,"value":"**The TF.js version used in this notebook is: ${tf.version.tfjs}**","pinned":false,"hidden":false,"mode":"md"},{"id":4,"value":"tf = require('@tensorflow/tfjs@4.0.0/dist/tf.min.js')","pinned":true,"hidden":false,"mode":"ojs"},{"id":788,"value":"import { pt } from \"@tmcw/pt\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":827,"value":"pc = d3.format(\".0%\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":852,"value":"import { toc } from \"@sugi2000/toc\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1041,"value":"MathJax = require(\"https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg.js\").catch(\n () => window[\"MathJax\"]\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1631,"value":"import { canvas } from \"6e3bcbbcfee2f7b6\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W34","stdlib":"1","titling":"auto"}