31:[] 2f:{"title":"TFJS - Fitting a curve to Synthetic data","cells":[{"id":0,"value":"md`# TFJS - Fitting a curve to Synthetic data`","pinned":false,"hidden":false,"mode":"ojs"},{"id":564,"value":"md`This is second article and it repeats tensorflow.js examples \n\nWhy Observable?\n\nBecause it's more fun to use & debug & learn & share new things here. \nIf you are interested, you can Take a look at the old articles \n\n\n1. [ Tensorflow - Core Concepts ](https://beta.observablehq.com/@bumbeishvili/tf-core-concepts) \n2. Current one - fitting a curve to synthetic data\n\n\n

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":531,"value":"md`If you don't want to read whole artice and just want to have fun, go ahead and\n\nclick ${label(\"start training\")} button`","pinned":false,"hidden":false,"mode":"ojs"},{"id":529,"value":"viewof startTrainingTopButton = button({value: \"Start Training\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":527,"value":"md`\n### TraininProgress - ${Math.floor(generatedPredictions.currentIteration/generatedPredictions.numIterations*100)+1} %\n\nIteration - ${generatedPredictions.currentIteration}\n\nTrue Coefficients \n${tex`y=-0.8x^3-0.2x^2+0.9x+0.5`} \n\nEstimated Coefficients \n${tex`y= ${val('a')} * x^3 ${val('b')}*x^2 ${val('c')}*x ${val('a')}`}\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":292,"value":"PredictionsTop = { \n \n plotDataAndPredictions('#predictingTop',trainingData.xs,trainingData.ys,generatedPredictions.newPredictions)\n return html`

`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":562,"value":"md`### First, load tensorflow`","pinned":false,"hidden":false,"mode":"ojs"},{"id":16,"value":"tf = require('@tensorflow/tfjs')","pinned":true,"hidden":false,"mode":"ojs"},{"id":77,"value":"md`We want to predict (${tex`a,b,c,d`} ) coefficients of cubic function\n\n${tex`y=ax^3+bx^2+cx+d`} \n\nSo, to check how tensorflow predicted, let's define true coefficients and keep them in mind\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":48,"value":"trueCoefficients = {\n return {a: -.8, b: -.2, c: .9, d: .5}\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":87,"value":"md`\nSo real function looks like this \n\n\n${tex`y=-0.8x^3-0.2bx^2+0.9x+0.5`} \n\n\nLet's generate data around this function using given coefficients`","pinned":false,"hidden":false,"mode":"ojs"},{"id":27,"value":"trainingData = generateData(200,trueCoefficients) ","pinned":true,"hidden":false,"mode":"ojs"},{"id":31,"value":"xs = trainingData.xs.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":39,"value":"ys = trainingData.ys.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":94,"value":"md`**Not** very **informative** output, is it?\n\nLet's plot it to see what it looks like visually `","pinned":false,"hidden":false,"mode":"ojs"},{"id":67,"value":"trainingDataPlot = { \n \n plotData('#trainingDataPlot',trainingData.xs,trainingData.ys)\n return html`

`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":114,"value":"md`So, we now can clearly see, there is some pattern \n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":119,"value":"md`## Step 1. Set up variables`","pinned":false,"hidden":false,"mode":"ojs"},{"id":128,"value":"md`Create variables which will hold our estimates, regarding coeffiecients \n\nInitialize them with random values`","pinned":false,"hidden":false,"mode":"ojs"},{"id":134,"value":"a = tf.variable(tf.scalar(Math.random()))","pinned":true,"hidden":false,"mode":"ojs"},{"id":173,"value":"a.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":139,"value":"b = tf.variable(tf.scalar(Math.random()))","pinned":true,"hidden":false,"mode":"ojs"},{"id":175,"value":"b.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":142,"value":"c = tf.variable(tf.scalar(Math.random()))","pinned":true,"hidden":false,"mode":"ojs"},{"id":177,"value":"c.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":145,"value":"d = tf.variable(tf.scalar(Math.random()))","pinned":true,"hidden":false,"mode":"ojs"},{"id":179,"value":"d.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":148,"value":"md`Let's toString them to see what inner values looks like \n\n\n**Estimated ones ** \n**a**=${a.toString()} \n**b**=${b.toString()} \n**c**=${c.toString()} \n**d**=${d.toString()} \n\nThey are clearly very different from true coefficients :) \nRemember them? \n\n\n**True ones ** \n**a**=${trueCoefficients.a.toString()} \n**b**=${trueCoefficients.b.toString()} \n**c**=${trueCoefficients.c.toString()} \n**d**=${trueCoefficients.d.toString()} \n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":182,"value":"md`## Step 2. Build a model`","pinned":false,"hidden":false,"mode":"ojs"},{"id":185,"value":"md`Write prediction function which calculates ${tex`y`} based on ${tex`x`}, taking earlier ${tex`y=ax^3+bx^2+cx+d`} function into account\n\n` ","pinned":false,"hidden":false,"mode":"ojs"},{"id":193,"value":"function predict(x){\n return tf.tidy(()=>{\n var ax3 = a.mul(x.pow(tf.scalar(3)));\n var bx2 = b.mul(x.square());\n var cx = c.mul(x)\n return ax3.add(bx2).add(cx).add(d); \n }) \n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":210,"value":"md`Generate initial predictions based on predicted coefficients`","pinned":false,"hidden":false,"mode":"ojs"},{"id":204,"value":"initialPredictions = predict(trainingData.xs)","pinned":true,"hidden":false,"mode":"ojs"},{"id":212,"value":"initialPredictions.toString()","pinned":false,"hidden":false,"mode":"ojs"},{"id":206,"value":"md`Let's plot this prediction graphically , along with trainingData `","pinned":false,"hidden":false,"mode":"ojs"},{"id":219,"value":"initialPredictionPlot = { \n \n plotDataAndPredictions('#initialPredictionPlot',trainingData.xs,trainingData.ys,initialPredictions)\n return html`

`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":226,"value":"md`As we can see, initial prediction is very poor `","pinned":false,"hidden":false,"mode":"ojs"},{"id":229,"value":"md`## Step 3. Train the Model`","pinned":false,"hidden":false,"mode":"ojs"},{"id":231,"value":"md`\nFor the model training we need to define\n* **Loss function** - measures how well polinomial fits data\n* **Optimiser** - To minimise output value of loss function\n* **Training loop** - iteratively runs optimizer for loss minimisation\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":242,"value":"md`#### Loss function\n\n
\nWe will be using **mean squared error (MSE)**\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":248,"value":"function loss(predictions, labels) {\n const meanSquareError = predictions.sub(labels).square().mean();\n return meanSquareError;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":258,"value":"md`#### Optimiser\n
\nWe'll use **Stochastic Gradient Descent(SGD)** \n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":263,"value":"Optimizer = {\n const learningRate = 0.5;\n return tf.train.sgd(learningRate); \n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":268,"value":"md`#### Training Loop\n
\nhere is a function which is responsible for looping\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":272,"value":"function train(xs, ys, numIterations=75){\n for(let iter=0; iter{\n const predsYs = predict(xs);\n return loss(predsYs,ys);\n })\n }\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":306,"value":"md`For the visual convenience, let's convert it to generator function \n\n(Thank you Observable!) `","pinned":false,"hidden":false,"mode":"ojs"},{"id":309,"value":"function* trainGenerator(xs, ys, numIterations=75,delay){\n for(let iter=0; iter{\n const predsYs = predict(xs);\n return loss(predsYs,ys);\n });\n \n var newPredictions = predict(xs);\n var result={\n currentIteration:iter,\n numIterations:numIterations,\n newPredictions:newPredictions,\n a:a,\n b:b,\n c:c,\n d:d\n }\n yield Promises.tick(delay||1000, result);\n }\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":447,"value":"md`Click start training and observe :)\n

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":410,"value":"viewof startTrainingButton = button({value: \"Start Training\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":330,"value":"md`\n### TraininProgress - ${Math.floor(generatedPredictions.currentIteration/generatedPredictions.numIterations*100)+1} %\n\nIteration - ${generatedPredictions.currentIteration}\n\nTrue Coefficients \n${tex`y=-0.8x^3-0.2x^2+0.9x+0.5`} \n\nEstimated Coefficients \n${tex`y= ${val('a')} * x^3 ${val('b')}*x^2 ${val('c')}*x ${val('a')}`}\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":541,"value":"Predictions = { \n \n plotDataAndPredictions('#predicting',trainingData.xs,trainingData.ys,generatedPredictions.newPredictions)\n return html`

`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":64,"value":"md`\n

\n# Some boring stuff`","pinned":false,"hidden":false,"mode":"ojs"},{"id":392,"value":"function val(c){\n var tensor = generatedPredictions[c];\n var val = tensor.dataSync()[0];\n var sign=\"\";\n if(val>0) {sign=\"+\";}\n return sign + Math.round(val*10000)/10000;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":19,"value":"function generateData(numPoints, coeff, sigma = 0.04) {\n return tf.tidy(() => {\n const [a, b, c, d] = [\n tf.scalar(coeff.a), tf.scalar(coeff.b), tf.scalar(coeff.c),\n tf.scalar(coeff.d)\n ];\n\n const xs = tf.randomUniform([numPoints], -1, 1);\n\n // Generate polynomial data\n const three = tf.scalar(3, 'int32');\n const ys = a.mul(xs.pow(three))\n .add(b.mul(xs.square()))\n .add(c.mul(xs))\n .add(d)\n // Add random noise to the generated data\n // to make the problem a bit more interesting\n .add(tf.randomNormal([numPoints], 0, sigma));\n\n // Normalize the y values to the range 0 to 1.\n const ymin = ys.min();\n const ymax = ys.max();\n const yrange = ymax.sub(ymin);\n const ysNormalized = ys.sub(ymin).div(yrange);\n\n return {\n xs, \n ys: ysNormalized\n };\n })\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":54,"value":"renderChart = require('vega-embed')","pinned":true,"hidden":false,"mode":"ojs"},{"id":56,"value":"async function plotData(container, xs, ys){\n const xvals = await xs.data();\n const yvals = await ys.data();\n\n const values = Array.from(yvals).map((y, i) => {\n return {'x': xvals[i], 'y': yvals[i]};\n });\n\n const spec = {\n '$schema': 'https://vega.github.io/schema/vega-lite/v2.json',\n 'width': 300,\n 'height': 300,\n 'data': {'values': values},\n 'mark': 'point',\n 'encoding': {\n 'x': {'field': 'x', 'type': 'quantitative'},\n 'y': {'field': 'y', 'type': 'quantitative'}\n }\n };\n\n return renderChart(container, spec, {actions: false});\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":60,"value":"async function plotDataAndPredictions(container, xs, ys, preds) {\n const xvals = await xs.data();\n const yvals = await ys.data();\n const predVals = await preds.data();\n\n const values = Array.from(yvals).map((y, i) => {\n return {'x': xvals[i], 'y': yvals[i], pred: predVals[i]};\n });\n\n const spec = {\n '$schema': 'https://vega.github.io/schema/vega-lite/v2.json',\n 'width': 300,\n 'height': 300,\n 'data': {'values': values},\n 'layer': [\n {\n 'mark': 'point',\n 'encoding': {\n 'x': {'field': 'x', 'type': 'ordinal'},\n 'y': {'field': 'y', 'type': 'quantitative'}\n }\n },\n {\n 'mark': 'line',\n 'encoding': {\n 'x': {'field': 'x', 'type': 'ordinal'},\n 'y': {'field': 'pred', 'type': 'quantitative'},\n 'color': {'value': 'tomato'}\n },\n }\n ]\n };\n\n return renderChart(container, spec, {actions: false});\n }\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":413,"value":"import {button} from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":423,"value":"trainingNotStarted = {\n startTrainingButton;\n startTrainingTopButton;\n return !this ;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":440,"value":"generatedPredictions ={\n startTrainingButton;\n startTrainingTopButton;\n return !trainingNotStarted? trainGenerator(trainingData.xs,trainingData.ys,200,100):\n trainGenerator(trainingData.xs,trainingData.ys,1,1)\n \n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":535,"value":"import {label} from \"@bumbeishvili/styles\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}