30:T4dc,md`# Introduction to JPEG For an university seminar i have to give a short introduction to the JPEG compression. To better understand how it works i coded some visualizations. Keep in mind that i am not an expert and that my code might be wrong or extremly unperformant. If you find mistakes or have suggestions just leave a comment. ### Inspiration A lot of the inspiration came from a [lecture](https://www.youtube.com/watch?v=7fhHQgu2OcY) (in German) by [Prof. Dr. Edmund Weitz](http://weitz.de/) who put together an [awesome webpage](http://weitz.de/dct/) which was a huge source of inspiration for me and really helped me understand it. ## color transformation First you have to transform the RGB data into another color space. The space that is used often with jpeg is [YCbCr](https://en.wikipedia.org/wiki/YCbCr#:~:text=YCbCr%2C%20Y%E2%80%B2CbCr%2C%20or,and%20red%2Ddifference%20chroma%20components.). Here Y is the luma component, Cb is the blue-difference and Cr is the red-difference chroma component. To illustrate this i used an awesome [example image](https://unsplash.com/photos/_NM6bXPqaCE) from unsplash (made by Geran de Klerk). To transform this image between the color spaces we can use a linear transformation given by`31:T43a,function drawCosine3d(scene, n, m) { const f = (u, v) => 4 * getDiscCosine(u, v)(n, m); const size = 5; for (let i = 0; i < 8; ++i) { for (let j = 0; j < 8; ++j) { const x = -18 + i * 5; const y = -18 + j * 5; const discCosine = getDiscCosine(i / 7, j / 7); const val = discCosine(n, m); const pgeo = new THREE.PlaneBufferGeometry(size, size, 20, 20); const pmat = new THREE.MeshBasicMaterial({ side: THREE.DoubleSide, color: cosValueToRgb(val) }); const plane = new THREE.Mesh(pgeo, pmat); plane.position.x = x; plane.position.z = y; plane.rotateX(Math.PI / 2); scene.add(plane); } } const paraGeo = new THREE.ParametricBufferGeometry( (u, v, dest) => dest.set(-20.5 + u * 40, 10 + f(u, v), -20.5 + v * 40), 40, 40 ); const paraMat = new THREE.MeshBasicMaterial({ color: 'rgb(0, 0, 255)', side: THREE.DoubleSide, transparent: true, opacity: 0.7, wireframe: true }); const para = new THREE.Mesh(paraGeo, paraMat); scene.add(para); }32:T57e,function drawCosineOneDim(n, context) { const { width, height } = context.canvas; const xStart = 70; const xEnd = width - 70; const yHeight = 300; const xAxis = [[xStart, yHeight], [xEnd, yHeight]]; const yAxis = [[xStart, 70], [xStart, height - 70]]; drawAxis(context, xAxis, yAxis); const scale = 200; const f = x => Math.cos(n * Math.PI * x); context.beginPath(); context.moveTo(xStart, yHeight - scale * f(0)); for (let i = 0; i <= 1; i += 0.001) { context.lineTo((1 - i) * xStart + i * xEnd, yHeight - scale * f(i)); } context.strokeStyle = 'rgb(0, 0, 255)'; context.stroke(); context.fillStyle = 'rgb(255, 0, 0)'; context.strokeStyle = 'rgb(255, 0, 0)'; const cosSamples = []; for (let i = 0; i <= 1; i += 1 / 7) { context.beginPath(); const p = [(1 - i) * xStart + i * xEnd, yHeight]; context.arc(...p, 5, 0, 2 * Math.PI, false); context.fill(); context.beginPath(); context.moveTo(...p); context.lineTo(p[0], yHeight - scale * f(i)); context.stroke(); cosSamples.push(f(i)); } const l = xEnd - xStart; const [boxW, boxH] = [l / 7, 20]; for (let i = 0; i < cosSamples.length; ++i) { context.fillStyle = cosValueToRgb(cosSamples[i]); context.fillRect(20 + i * boxW, height - 30, boxW, boxH); context.strokeStyle = 'rgb(255,0,0)'; context.strokeRect(20 + i * boxW, height - 30, boxW, boxH); } }33:[{"id":"ec4ea309cc6780fa6b243cb0f420e26b44d56917d64ffaebdae455e16b5b482f54acb8a92e4c9456f1a36b849d4f5e2c688bc33a40bb9bd1da36a4479d56865d","href":"https://static.observableusercontent.com/files/ec4ea309cc6780fa6b243cb0f420e26b44d56917d64ffaebdae455e16b5b482f54acb8a92e4c9456f1a36b849d4f5e2c688bc33a40bb9bd1da36a4479d56865d","name":"geran-de-klerk_200.jpg","type":"image/jpeg","lastModified":1610372697966.005,"size":78203}] 2e:{"title":"Introduction to JPEG","cells":[{"id":0,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":29,"value":"html`\n${tex`\n \\begin{bmatrix}\n Y \\\\ Cb \\\\ Cr\n \\end{bmatrix} \\approx\n \\begin{bmatrix}\n 0 \\\\ 128 \\\\ 128\n \\end{bmatrix} + \n \\begin{bmatrix}\n 0.299 & 0.587 & 0.114 \\\\\n -0.168736 & -0.331264 & 0.5 \\\\\n 0.5 & -0.418688 & -0.081312\n \\end{bmatrix} \\cdot\n \\begin{bmatrix}\n R \\\\ G \\\\ B\n \\end{bmatrix}`}\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":124,"value":"md`\nThis formula is taken from [here](https://de.wikipedia.org/wiki/JPEG#Farbmodellumrechnung) (it might not match the spec).\n\nIn the following graphic i ploted the three sperate channels. From left to right we have Y, Cb, Cr.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":89,"value":"{\n const context = DOM.context2d(width, 200);\n drawYCbCrToCanvas(yCbCrImgData, context);\n yield context.canvas;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":127,"value":"md`In the original algorihm, all three channels are handled seperately. But for simplicity i will only concentrate on the Y channel. As the procedure is the same for all channels.\n\n## split the image\n\nTo speed up computation, the image is now split up into 8x8 blocks. I used an (very) naive method to do this. Therefore i can only use square images where the width and height is a multiple of 8. But in practise one could use some padding and a smarter method to use arbitrary image sizes (homework for you if you want).`","pinned":false,"hidden":false,"mode":"ojs"},{"id":157,"value":"{\n const context = DOM.context2d(280, 280);\n drawImg8x8Blocks(blocks, context, yChannel.width, yChannel.height);\n yield html`${context.canvas}`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":185,"value":"md`\nnow every 8x8 block is handled seperately. This would be nice to handle in parallel (e.g. with web workers if you use js) but for simplicity i will handle them synchronous, which will impact performance but will make my life easier.\n\n## discrete cosine\nbefore we handle the blocks, we will make a short excourse to the discrete cosine. The main idea here is to sample discrete (meaning finite many) points from a cosine curve and interpret those values as gray values. Below you can see a cosine curve and 8 equidistant points that sample from it. Those values are in the range [-1, 1] and are mapped to [0, 255] and ploted as gray values. \n\nThe function you can see below is\n${html`\n${tex`\n \\begin{aligned}\n f &: \\mathbb{R} \\to \\R \\\\\n x & \\longmapsto \\cos(${n} \\cdot x)\n \\end{aligned}\n `}\n`}\n(which is scaled to better illustrate it)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":235,"value":"viewof n = slider({\n value: 4,\n min: 0,\n max: 8,\n step: 1,\n title: 'n (frequency)',\n ...sliderConfig\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":200,"value":"{\n const height = 600 / dpr;\n const w = width / dpr;\n const context = DOM.context2d(w, height);\n context.fillStyle = `rgba(200, 200, 200, 0.2)`;\n context.fillRect(0, 0, w, height);\n context.scale(1 / dpr, 1 / dpr);\n drawCosineOneDim(n, context, w);\n yield context.canvas;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":280,"value":"md`\nAs you can see, a high value of the cosine function corresponds to a bright gray and a low cosine value to a dark gray. Now we can apply the same concept to onther function\n\n${html`\n \n ${tex`\n \\begin{aligned}\n g &: \\mathbb{R}^2 \\to \\mathbb{R} \\\\\n (x, y) &\\longmapsto \\cos(n\\pi x)\\cos(m\\pi y)\n \\end{aligned}\n `}\n \n `}\nwhere ${tex`m,n \\in [0,8]`}. Note that in practice a slightly [modified function](https://en.wikipedia.org/wiki/JPEG#Discrete_cosine_transform) is used, but for simplicity i will use g.\nIf we think about x,y as a plane and plot ${tex`g(x,y)`} in the z direction we get a surface.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":334,"value":"viewof values = inputsGroup([\n [\n slider({\n value: 2,\n min: 0,\n max: 8,\n step: 1,\n title: 'n',\n ...sliderConfig\n }),\n slider({\n value: 3,\n min: 0,\n max: 8,\n step: 1,\n title: 'm',\n ...sliderConfig\n })\n ]\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":292,"value":"canvas3d = DOM.canvas(width, 500)","pinned":false,"hidden":false,"mode":"ojs"},{"id":381,"value":"md`\nPlay around with the sliders to see which 8x8 patterns are possible. \n\nNow we can create a table for all possible 8x8 patterns and plot them as a table. Here the x frequency increases in x direction of the table and the y frequency increases in the y direction. Resulting in the following table\n\n(there might be a problem with the device pixel ratio in the next graphic)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":387,"value":"{\n const width = 600 / dpr;\n const height = 600 / dpr;\n const context = DOM.context2d(width, height);\n context.fillStyle = `rgba(200, 200, 200, 0.2)`;\n context.fillRect(0, 0, width, height);\n drawCosineTable(context);\n\n // highlight current pattern\n const [[n, m]] = values;\n context.strokeStyle = 'rgb(0,0,255)';\n context.lineWidth = 4 / dpr;\n const offset = 20;\n const [stepX, stepY] = [\n (width / dpr - 2 * offset) / 8,\n (height / dpr - 2 * offset) / 8\n ];\n\n context.strokeRect(\n offset + n * stepX,\n offset + m * stepY,\n stepX - 10,\n stepY - 10\n );\n\n yield html`${context.canvas}`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":421,"value":"md`\nWe can now represent every black and white 8x8 block as a linear combination of those blocks. Lets illustrate that with an example pattern that we want to reconstruct from the patterns above. (Note that we scaled the 8x8 pixel pattern here to better see the results).\n\nLeft is the original Pattern, in the middle you can see the coefficients of the linear combination that result in the reconstructed image on the right. A dark gray means that this coefficient is used in the reconstruction.\n\nPlay around with the slider below. If you include all coefficients you will get precisely the original pattern. But note that you can leave out some of the coefficents that have a small absolute value and get approximately the same pattern.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":473,"value":"viewof smallestCoeff = slider({\n value: 0.4,\n min: 0,\n max: 2,\n step: 0.0001,\n title: 'only Coefficients with absolute value bigger than this are used',\n ...sliderConfig\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":428,"value":"{\n const height = 300;\n const width = 300;\n\n // plot example pattern\n const contextExample = DOM.context2d(width, height);\n contextExample.scale(1 / dpr, 1 / dpr);\n contextExample.fillStyle = `rgba(200, 200, 200, 0.2)`;\n contextExample.fillRect(0, 0, width, height);\n drawTile(exampleTile, contextExample);\n\n // reconstruct the pattern\n const coeffs = getCoeffs(exampleTile);\n const reconstruction = reconstruct(coeffs, smallestCoeff);\n\n // plot the coefficients\n const contextCoeffs = DOM.context2d(width, height);\n contextCoeffs.scale(1 / dpr, 1 / dpr);\n drawCoeffTable(coeffs, smallestCoeff, contextCoeffs);\n\n const contextReconst = DOM.context2d(width, height);\n contextReconst.scale(1 / dpr, 1 / dpr);\n contextReconst.fillStyle = `rgba(200, 200, 200, 0.2)`;\n contextReconst.fillRect(0, 0, width, height);\n drawTile(reconstruction, contextReconst);\n\n yield html`\n

\n ${contextExample.canvas}\n ${contextCoeffs.canvas}\n ${contextReconst.canvas}\n

\n `;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":512,"value":"md`## putting it together\n\nNow we are able to reconstruct every 8x8 pixel block as a linear combination of the discrete cosine basis functions. This method can now be applied to our original image, which we sliced into 8x8 blocks.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":539,"value":"viewof smallestCoeffRes = slider({\n value: 12,\n min: 0,\n max: 20,\n step: 0.0001,\n title: 'only Coefficients with absolute value bigger than this are used',\n ...sliderConfig\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":516,"value":"{\n const width = 200;\n const height = 200;\n const context = DOM.context2d(width, height);\n context.fillStyle = `rgba(200, 200, 200, 0.2)`;\n context.fillRect(0, 0, width, height);\n\n const compressed = compressBlocks(blocks, smallestCoeffRes);\n const compressedImg = imgDataFrom8x8Blocks(compressed, width, height);\n drawChannel(compressedImg, context, [0, 0]);\n\n yield html`${context.canvas}`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":560,"value":"md` ## next steps\n\nThere are now still a few steps required to get the compressed jpeg image, but i will only briefly mention them here.\n\n#### Quantization\n\nThis uses the knowledge from above. We don't need all the frequencies to reconstruct an approximation to the original image. Instead we can use a Quantization (dividing the coefficients by a constant and rounding the result) to get rid of coefficients with small absolute value.\n\n#### Entropy Encoding\n\nThe Quantization step results in a sparse matrix (lot of entries are zero). This can easily compressed using run length encoding. If you want to know more please visit the [wikipedia page](https://en.wikipedia.org/wiki/JPEG#Entropy_coding) \n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":520,"value":"function compressBlocks(blocks8x8, precision) {\n const res = [];\n for (let block of blocks8x8) {\n const coeffs = getCoeffs(block);\n const reconst = reconstruct(coeffs, precision);\n res.push(reconst);\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":457,"value":"function reconstruct(coeffs, precision = 0.2) {\n const basis = [];\n for (let i = 0; i < 8; ++i) {\n for (let j = 0; j < 8; ++j) {\n const discCosMat = getDiscCosineMat(i, j);\n basis.push(discCosMat);\n }\n }\n\n let val = zeros(8, 8);\n for (let i = 0; i < coeffs.length; ++i) {\n if (Math.abs(coeffs[i]) >= precision) {\n // coeff_i * A_i\n const nextTerm = scalarMatMul(coeffs[i], basis[i]);\n val = matAdd(val, nextTerm);\n }\n }\n\n return val;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":464,"value":"function zeros(n, m) {\n const res = [];\n for (let i = 0; i < n; ++i) {\n const row = [];\n for (let j = 0; j < m; ++j) {\n row.push(0);\n }\n res.push(row);\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":461,"value":"function scalarMatMul(scalar, mat) {\n const res = [];\n for (let row of mat) {\n const nextRow = [];\n for (let val of row) {\n nextRow.push(scalar * val);\n }\n res.push(nextRow);\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":458,"value":"function matAdd(mat1, mat2) {\n const res = [];\n for (let i = 0; i < mat1.length; ++i) {\n const nextRow = [];\n for (let j = 0; j < mat1[i].length; ++j) {\n const val = mat1[i][j] + mat2[i][j];\n nextRow.push(val);\n }\n res.push(nextRow);\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":447,"value":"function getCoeffs(resTile) {\n const b = matToVec(resTile);\n const A = [];\n for (let i = 0; i < 8; ++i) {\n for (let j = 0; j < 8; ++j) {\n const discCosMat = getDiscCosineMat(i, j);\n const asVec = matToVec(discCosMat);\n A.push(asVec);\n }\n }\n return linear.solve(A, b);\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":451,"value":"function getDiscCosineMat(n, m) {\n const res = [];\n for (let i = 0; i < 8; ++i) {\n const col = [];\n for (let j = 0; j < 8; ++j) {\n const discCosine = getDiscCosine(i / 7, j / 7);\n const val = discCosine(n, m);\n col.push(val);\n }\n res.push(col);\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":448,"value":"function matToVec(mat) {\n const res = [];\n for (let row of mat) {\n for (let val of row) {\n res.push(val);\n }\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":431,"value":"function drawTile(mat, context) {\n const [w, h] = [context.canvas.width, context.canvas.height];\n const [stepX, stepY] = [w / 8, h / 8];\n for (let i = 0; i < 8; ++i) {\n for (let j = 0; j < 8; ++j) {\n const x_i = i * stepX;\n const y_i = j * stepY;\n const val = mat[i][j];\n context.fillStyle = cosValueToRgb(val);\n context.fillRect(x_i, y_i, stepX, stepY);\n }\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":357,"value":"setup3d = setupCosine3d(canvas3d)","pinned":false,"hidden":false,"mode":"ojs"},{"id":309,"value":"{\n const [[n, m]] = values;\n let [camera, scene, renderer, controls] = setupCosine3d(canvas3d);\n drawCosine3d(scene, n, m);\n const render = () => {\n window.requestAnimationFrame(render);\n controls.update();\n renderer.render(scene, camera);\n };\n render();\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":322,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":391,"value":"function drawCosineTable(context) {\n const { width, height } = context.canvas;\n\n const offset = 20;\n const [stepX, stepY] = [\n (width / dpr - 2 * offset) / 8,\n (height / dpr - 2 * offset) / 8\n ];\n for (let n = 0; n < 8; ++n) {\n context.fillStyle = \"rgb(0,0,0)\";\n context.fillText(n, 1.5 * offset + n * stepX, offset / 1.5);\n context.fillText(n, offset / 3, 2 * offset + n * stepY);\n for (let m = 0; m < 8; ++m) {\n const x = offset + n * stepX;\n const y = offset + m * stepY;\n draw8x8Tile(n, m, x, y, stepX - 10, stepY - 10, context);\n }\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":392,"value":"function draw8x8Tile(n, m, x, y, w, h, context) {\n const [stepX, stepY] = [w / 8, h / 8];\n for (let i = 0; i < 8; ++i) {\n for (let j = 0; j < 8; ++j) {\n const x_i = i * stepX;\n const y_i = j * stepY;\n const discCosine = getDiscCosine(i / 7, j / 7);\n const val = discCosine(n, m);\n context.fillStyle = cosValueToRgb(val);\n context.fillRect(x + x_i, y + y_i, stepX, stepY);\n }\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":290,"value":"function drawCoeffTable(coeffs, precision, context) {\n const { width, height } = context.canvas;\n context.font = \"10px sans-serif\";\n const offset = 5;\n const [stepX, stepY] = [(width - 2 * offset) / 8, (height - 2 * offset) / 8];\n const [w, h] = [stepX - 3.5, stepY - 3.5];\n context.textAlign = 'center';\n for (let i = 0; i < 8; ++i) {\n for (let j = 0; j < 8; ++j) {\n const x = offset + i * stepX;\n const y = offset + j * stepY;\n const coeff = coeffs[i * 8 + j];\n if (Math.abs(coeff) >= precision) {\n context.fillStyle = 'rgba(100, 100, 100, 0.8)';\n context.fillRect(x, y, w, h);\n }\n context.strokeRect(x, y, w, h);\n context.stroke();\n context.fillStyle = 'rgb(0,0,0)';\n context.fillText(coeff.toFixed(3), x + stepX / 2, y + stepY / 2);\n }\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":323,"value":"function getDiscCosine(x, y) {\n return (n, m) => {\n return Math.cos(n * x * Math.PI) * Math.cos(m * y * Math.PI);\n };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":205,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":203,"value":"function cosValueToRgb(val) {\n return `rgb(${Math.round(((val + 1) * 255) / 2)}, ${Math.round(\n ((val + 1) * 255) / 2\n )}, ${Math.round(((val + 1) * 255) / 2)})`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":197,"value":"function drawAxis(context, xAxis, yAxis) {\n for (let ax of [xAxis, yAxis]) {\n const [axisStart, axisEnd] = ax;\n context.beginPath();\n context.strokeStyle = 'rgb(0,0,0)';\n context.moveTo(...axisStart);\n context.lineTo(...axisEnd);\n context.stroke();\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":146,"value":"function getYChannel(yCbCr) {\n const context = document.createElement('canvas').getContext('2d');\n const res = context.createImageData(yCbCr);\n for (let i = 0; i < yCbCr.data.length; i += 4) {\n res.data[i] = yCbCr.data[i];\n res.data[i + 1] = yCbCr.data[i];\n res.data[i + 2] = yCbCr.data[i];\n res.data[i + 3] = 255;\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":144,"value":"function drawImg8x8Blocks(blocks8x8, context, width, height) {\n const asImgDatas = [];\n for (let block of blocks8x8) {\n const flat = _.flattenDeep(block);\n const asImgData = context.createImageData(8, 8);\n for (let i = 0; i < flat.length; ++i) {\n const val = flat[i];\n asImgData.data[4 * i + 0] = val;\n asImgData.data[4 * i + 1] = val;\n asImgData.data[4 * i + 2] = val;\n asImgData.data[4 * i + 3] = 255;\n }\n asImgDatas.push(asImgData);\n }\n\n for (let j = 0; j < height / 8; ++j) {\n for (let i = 0; i < width / 8; ++i) {\n drawChannel(asImgDatas[i + (j * width) / 8], context, [\n i * 8 * 2,\n j * 8 * 2\n ]);\n }\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":142,"value":"function imgDataFrom8x8Blocks(blocks8x8, width, height) {\n const [numBlocksW, numBlocksH] = [\n Math.ceil(width / 8),\n Math.ceil(height / 8)\n ];\n const rows = [];\n for (let i = 0; i < numBlocksH; ++i) {\n const sliceInd = numBlocksW * i;\n const blockRows = blocks8x8.slice(sliceInd, sliceInd + numBlocksW);\n const blockRowsZip = _.zip(...blockRows);\n rows.push(_.flattenDeep(blockRowsZip));\n }\n const data = _.flattenDeep(rows);\n const context = document.createElement('canvas').getContext('2d');\n const res = context.createImageData(width, height);\n\n for (let i = 0; i < data.length; ++i) {\n const val = data[i];\n res.data[4 * i + 0] = val;\n res.data[4 * i + 1] = val;\n res.data[4 * i + 2] = val;\n res.data[4 * i + 3] = 255;\n }\n\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":138,"value":"function imgDataInto8x8Blocks(imgData) {\n // WARNING / TODO only supports square images of size 8n x 8n\n // also very naive implementation\n const { data, width, height } = imgData;\n const firstChannelData = data.filter((d, i) => i % 4 === 0);\n\n const rows = _.chunk(firstChannelData, width);\n const rowBlocks = [];\n for (let row of rows) {\n const chunked = _.chunk(row, 8);\n rowBlocks.push(chunked);\n }\n\n const numBlocksH = Math.ceil(height / 8);\n const blocks = [];\n for (let j = 0; j < numBlocksH; ++j) {\n const blocksW = [];\n for (let i = 0; i < rowBlocks[0].length; ++i) {\n const row = [];\n for (let ix = j * 8; ix < j * 8 + 8; ++ix) {\n row.push(rowBlocks[ix][i]);\n }\n blocksW.push(row);\n }\n blocks.push(...blocksW);\n }\n\n return blocks;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":97,"value":"function drawYCbCrToCanvas(imgdata, context) {\n const y = context.createImageData(imgdata);\n const cb = context.createImageData(imgdata);\n const cr = context.createImageData(imgdata);\n for (let i = 0; i < imgdata.data.length; i += 4) {\n const yData = imgdata.data[i];\n const cbData = imgdata.data[i + 1];\n const crData = imgdata.data[i + 2];\n y.data[i] = yData;\n y.data[i + 1] = yData;\n y.data[i + 2] = yData;\n y.data[i + 3] = 255;\n cb.data[i] = 128;\n cb.data[i + 1] = 128;\n cb.data[i + 2] = cbData;\n cb.data[i + 3] = 255;\n cr.data[i] = crData;\n cr.data[i + 1] = 128;\n cr.data[i + 2] = 128;\n cr.data[i + 3] = 255;\n }\n drawChannel(y, context, [150, 0]);\n drawChannel(cb, context, [360, 0]);\n drawChannel(cr, context, [570, 0]);\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":95,"value":"function drawChannel(data, context, pos) {\n const tmpCan = document.createElement('canvas');\n tmpCan.width = data.width;\n tmpCan.height = data.height;\n const tmpContext = tmpCan.getContext('2d');\n tmpContext.putImageData(data, 0, 0);\n const img = document.createElement('img');\n img.src = tmpCan.toDataURL();\n img.onload = () => {\n context.drawImage(img, ...pos);\n };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":78,"value":"function rgbToYCbCr(rgb) {\n // transform rgb imgData to YCbCr imgData\n const res = DOM.context2d(rgb.width, rgb.height).createImageData(rgb);\n for (let i = 0; i < rgb.data.length; i += 4) {\n const [r, g, b] = [rgb.data[i], rgb.data[i + 1], rgb.data[i + 2]];\n const y = 0.299 * r + 0.587 * g + 0.114 * b;\n const cb = 128 + -0.168736 * r + -0.331264 * g + 0.5 * b;\n const cr = 128 + 0.5 * r + -0.331264 * g + -0.081312 * b;\n res.data[i + 0] = y;\n res.data[i + 1] = cb;\n res.data[i + 2] = cr;\n res.data[i + 3] = rgb.data[i + 3];\n }\n return res;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":51,"value":"function imgDataFromImg(img) {\n const context = DOM.context2d(img.width, img.height);\n context.drawImage(img, 0, 0, img.width, img.height);\n return context.getImageData(0, 0, img.width, img.height);\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":424,"value":"exampleTile = [\n [0, 29, 255, 135, 255, 255, 255, 0],\n [255, 0, 255, 255, 255, 255, 0, 255],\n [235, 255, 0, 0, 0, 0, 255, 195],\n [155, 255, 0, 255, 255, 0, 255, 215],\n [255, 255, 0, 255, 205, 0, 250, 255],\n [255, 255, 0, 0, 0, 0, 255, 255],\n [255, 0, 255, 255, 145, 255, 0, 255],\n [0, 25, 255, 19, 255, 255, 255, 0]\n].map(row => row.map(v => (v - 127) / 127))","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"file = FileAttachment(\"geran-de-klerk_200.jpg\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":151,"value":"rgbImgData = imgDataFromImg(file)","pinned":false,"hidden":false,"mode":"ojs"},{"id":153,"value":"yCbCrImgData = rgbToYCbCr(rgbImgData)","pinned":false,"hidden":false,"mode":"ojs"},{"id":160,"value":"yChannel = getYChannel(yCbCrImgData)","pinned":false,"hidden":false,"mode":"ojs"},{"id":523,"value":"blocks = imgDataInto8x8Blocks(yChannel)","pinned":false,"hidden":false,"mode":"ojs"},{"id":587,"value":"dpr = window.devicePixelRatio","pinned":false,"hidden":false,"mode":"ojs"},{"id":135,"value":"_ = require('lodash@4.17.20/lodash.js')","pinned":false,"hidden":false,"mode":"ojs"},{"id":303,"value":"function setupCosine3d(canvas) {\n const { width, height } = canvas;\n const camera = new THREE.PerspectiveCamera(45, width / height, 0.001, 1000);\n camera.position.z = 60;\n camera.position.y = 30;\n camera.lookAt(new THREE.Vector3(0, 0, 0));\n const scene = new THREE.Scene();\n const renderer = new THREE.WebGLRenderer({\n canvas,\n antialias: true,\n autoClear: true\n });\n renderer.setClearColor(0xeeeeee);\n const controls = new THREE.OrbitControls(camera, renderer.domElement);\n return [camera, scene, renderer, controls];\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":295,"value":"THREE = {\n const THREE = (window.THREE = await require(\"three@0.99.0/build/three.min.js\"));\n await require(\"three@0.99.0/examples/js/controls/OrbitControls.js\").catch(\n () => {}\n );\n return THREE;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":453,"value":"linear = require('https://bundle.run/linear-solve@1.2.1')","pinned":false,"hidden":false,"mode":"ojs"},{"id":274,"value":"sliderConfig = {\n return {\n theme: 'default-round',\n background: {\n type: 'double',\n colors: ['#7295FF', '#efefef']\n }\n };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":271,"value":"import { slider } from '@bartok32/diy-inputs'","pinned":false,"hidden":false,"mode":"ojs"},{"id":331,"value":"import { inputsGroup } from '@bumbeishvili/input-groups'","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W33","stdlib":"1","titling":"auto"}