31:Td21,md`Brief sketch of the idea: **A steppable visualization of JS execution**. Scopes and object references and the like. Aimed at teaching purposes only, not correctness. Multiple levels of detail: expression-detail vs statement-detail, and possibly max depth configs for jumping into function calls / certain block statements... - Necessarily means a full computation of code, because e.g. it has to figure out the actual number of execution iterations in a for loop. - Straight-to-the-point implementation: babel AST output node types are mapped pretty much directly onto execution mechanisms. These mechanisms are generator functions, which are just functions that operate on an execution context, but because they're generators enable an elegant report-yielding API while stepping through the execution process of the node in question. - These intermediate (yielded) reports can be anything, but for now are objects of the shape \`{ context, info: { summary, pre, detail } }\`. The \`pre\` denotes a pre-evaluation step, which can be essential for visualization purposes, the \`summary\` a textual description, and the \`detail\` a number indicating whether it's a expression/statement. \`\`\`ts type NodeExecutor = ( node: N, context: Context, { report: Report => void; ...params } ) => Generator \`\`\` For example, the \`NumericLiteral\` executor looks like this: \`\`\`js function* (node, context, { report }) { yield report(context, { node, detail: 2, summary: "evaluated number " + node.value }); return node.value; } \`\`\` - Because Observable cells treat generators first-class, you get to see the "blip" of execution when you eval a cell :D How fun is that? - Not making any efforts to make these generator functions operate on the context in an immutable way, because that can just as well be achieved by a wrapper that, while stepping through the obtainer generator, use proxying library like Immer, or just deep copying, to record a history. - _(possibly a temporary choice)_ Normalized execution context: \`\`\`ts type Context = { scopes: Scope[]; objects: Obj[]; currentScope: scope_ref; // index } \`\`\` Previous work: - http://pythontutor.com/visualize.html - https://tylermcginnis.com/javascript-visualizer/ [also see the [github repo](https://github.com/tylermcginnis/javascriptvisualizer)] - http://latentflip.com/loupe/ - https://github.com/brendan-w/execute-js - [interpreter] https://github.com/NeilFraser/JS-Interpreter **Why? Does it improve?** Not sure if it'll improve. Tyler's viz is just excellent, and beautiful as well. It has closure scopes too, which Pythontutor does not. So probs my version is going to be _strictly worse_ than Tyler's :P (In correctness and looks.) But there are a few ways in which I might be adding: - ES6 support (But then because my app is educational, I'm actually stripping things like \`var\`... So our syntaxes will be non-overlapping, haha...) - More configurable in what's included in execution history. I generate "flat" report objects to history, which can be filtered all kinds of ways, incl. detail but also including pre/post execution, specific nodes, etc. - Ah, and where Pythontutor does not have closure scopes, Tyler's viz does not have an object reference graph. I'll have both :D`32:T437,function drawContext(context) { const { scopes, objects, currentScope } = context; function drawValue(value) { if (typeof value === "string") return `"${value}"`; if (typeof value === "object" && "object_ref" in value) return `object #${value.object_ref}`; return value; } function drawVar({ name, kind, value }) { return `

${kind} ${name || ""} ${value !== undefined ? `${kind === "return" ? `` : `= `}${drawValue(value)}` : ``}

`; } function drawScope({ parent, children, variables }, ref) { const current = ref === currentScope; const vars = Object.values(variables); return `

${vars.map(drawVar).join("")} ${variables[Return.symbol] ? drawVar(variables[Return.symbol]) : ``}

${children.length > 0 ? `

${children.map(ref => drawScope(scopes[ref], ref)).join("")}

` : ``}

`; } return drawScope(scopes[0], 0); }33:T2d23,execute = { const executors = {}; function makeSummaryReporter () { let i = 0; return function (context, info) { return `[${i++}] ${info.pre ? "pre-" : ""} ${info.summary}`; }; } function* exec(node, context = makeInitialContext(), options = { report: makeSummaryReporter() }) { const node_executor = executors[node.type]; if (!node_executor) { console.log(node); throw new NotImplemented(`No executor for AST nodes of type: ${node.type}`); } return yield* node_executor(node, context, options); } function applyOperator(op, a, b) { if (op === "+") return a + b; if (op === "-") return a - b; if (op === "*") return a * b; if (op === "&&") return a && b; if (op === "||") return a || b; throw new NotImplemented(`operator not implemented: ${op}`); } executors.File = function* (node, context, { report }) { yield* exec(node.program, context, { report }); }; executors.Program = function* (node, context, { report }) { // TODO actually extract function definitions etc. for (const statement of node.body) { yield* exec(statement, context, { report }); } }; executors.BlockStatement = function* (node, context, { report, skipScope }) { // TODO make new scope instead if `skip_scope` set for (const statement of node.body) { yield* exec(statement, context, { report }); } }; executors.ReturnStatement = function* (node, context, { report }) { yield report(context, { node, pre: true, summary: `return`, detail: 1 }); const ret = new Return(yield* exec(node.argument, context, { report })); yield report(context, { node, summary: `return`, detail: 1 }); throw ret; }; function* invokeFunction ({ definingScope, params, body }, args, context, { report }) { // Create an execution scope const executionScope = { parent: definingScope, children: [], variables: {} }; const scope_ref = context.scopes.push(executionScope) - 1; context.scopes[definingScope].children.push(scope_ref); // Populate with given arguments for (const [i, param] of params.entries()) { if (param.type !== "Identifier") throw new NotImplemented(`Sorry, can't deal with ${param.type} params yet :(`); executionScope.variables[param.name] = { name: param.name, kind: "param", value: args[i] }; } let result; if (body.type === "BlockStatement") { result = yield* Return.from(function* () { return yield* exec(body, { ...context, currentScope: scope_ref }, { report, skipScope: true }); }); } else { result = yield* exec(body, { ...context, currentScope: scope_ref }, { report }); } executionScope.variables[Return.symbol] = { kind: "return", value: result }; return result; }; executors.CallExpression = function* (node, context, { report }) { const { callee, arguments: argnodes } = node; yield report(context, { node, pre: true, summary: `evaluate call`, detail: 2 }); // Second, lookup the function const fnValue = yield* exec(callee, context, { report }); if (!("object_ref" in fnValue)) throw new RuntimeError(`Cannot call non-function #1`); const fn = context.objects[fnValue.object_ref]; if (!fn || fn.type !== "function") throw new RuntimeError(`Cannot call non-function #2`); // First, evaluate arguments const args = []; for (const argnode of argnodes) { args.push(yield* exec(argnode, context, { report })); } // Finally, invoke the function yield report(context, { node, pre: true, summary: `invoke function`, detail: 2 }); const ret = yield* invokeFunction(fn, args, context, { report }); yield report(context, { node, summary: `evaluated call`, detail: 2 }); return ret; }; executors.FunctionDeclaration = function* (node, context, { report }) { if (node.async || node.generator) throw new NotSupported(`Async and generator functions are not supported`); if (!node.id || node.id.type !== "Identifier") throw new RuntimeError(`should not happen`); yield report(context, { node, pre: true, summary: `declare function ${node.id.name}`, detail: 1 }); const scope = context.scopes[context.currentScope]; const fn = { type: "function", definingScope: context.currentScope, params: node.params, body: node.body }; const object_ref = context.objects.push(fn) - 1; scope.variables[node.id.name] = { name: node.id.name, kind: "function", value: { object_ref } }; yield report(context, { node, summary: `declared function ${node.id.name}`, detail: 1 }); }; executors.ObjectProperty = function* (node, context, { report, object_ref }) { const { key } = node; if (key.type !== "Identifier") throw new NotImplemented(`Computed or otherwise property keys are sadly not implemented yet :(`); const obj = context.objects[object_ref]; const value = yield* exec(node.value, context, { report }); obj.properties[key.name] = { name: key.name, kind: "property", value }; }; executors.ObjectExpression = function* (node, context, { report }) { yield report(context, { node, pre: true, summary: `evaluate obj expression`, detail: 2 }); const obj = { type: "object", properties: {} }; const object_ref = context.objects.push(obj) - 1; for (const property of node.properties) { yield* exec(property, context, { report, object_ref }); } yield report(context, { node, summary: `evaluated obj expression`, detail: 2 }); return { object_ref }; }; executors.ArrowFunctionExpression = function* (node, context, { report }) { if (node.async || node.generator) throw new NotSupported(`Async and generator functions are not supported`); const scope = context.scopes[context.currentScope]; const fn = { type: "function", definingScope: context.currentScope, params: node.params, body: node.body }; const object_ref = context.objects.push(fn) - 1; yield report(context, { node, summary: `evaluated arrow function`, detail: 2 }); return { object_ref }; }; executors.BinaryExpression = function* (node, context, { report }) { const { left, right } = node; yield report(context, { node, pre: true, summary: `evaluate binary expression`, detail: 2 }); const leftValue = yield* exec(left, context, { report }); if (node.operator === "&&" && !leftValue) return leftValue; if (node.operator === "||" && leftValue) return leftValue; const rightValue = yield* exec(right, context, { report }); yield report(context, { node, summary: `evaluated binary expression`, detail: 2 }); return applyOperator(node.operator, leftValue, rightValue); }; // (a) lookup identifier function* _LookupIdentifier (node, context, { report }) { const defining_scope_ref = lookup(context, node.name, context.currentScope); if (defining_scope_ref === null) throw new RuntimeError(`variable ${node.name} is not defined [lookup identifier]`); yield report(context, { node, summary: `looked up ${node.name}`, detail: 2 }); return { scope_ref: defining_scope_ref, name: node.name, site: context.scopes[defining_scope_ref].variables[node.name] }; }; // (b) evaluate identifier as value (more common) executors.Identifier = function* (node, context, { report }) { if (node.name === "undefined") { yield report(context, { node, summary: `evaluated undefined`, detail: 2 }); return undefined; } else { const defining_scope_ref = lookup(context, node.name, context.currentScope); if (defining_scope_ref === null) { throw new RuntimeError(`variable ${node.name} is not defined [eval identifier]`); } yield report(context, { node, summary: `looked up ${node.name}`, detail: 2 }); return context.scopes[defining_scope_ref].variables[node.name].value; } }; executors.MemberExpression = function* (node, context, { report }) { const { computed, object, property } = node; const obj = yield* exec(object, context, { report }); let propertyName; if (computed) { propertyName = yield* exec(); } else { if (property.type !== "Identifier") throw "should not happen"; propertyName = property.name; } const defining_object_ref = lookupProperty(context, propertyName); if (defining_object_ref === null) throw new RuntimeError(`property ${node.name} not found`); yield report(context, { node, summary: `looked up property ${propertyName}`, detail: 2 }); return { object_ref: defining_object_ref, name: propertyName, site: context.objects[defining_object_ref].properties[propertyName] }; }; executors.AssignmentExpression = function* (node, context, { report }) { const { left, right } = node; // yield report(context, { node, pre: true, summary: `assign ${left.name}`, detail: 2 }); const { site } = (yield* _LookupIdentifier(left, context, { report })); site.value = yield* exec(right, context, { report }); // yield report(context, { node, summary: `assigned ${left.name}`, detail: 2 }); }; executors.ExpressionStatement = function* (node, context, { report }) { yield report(context, { node, pre: true, summary: `compute expression statement`, detail: 1 }); yield* exec(node.expression, context, { report }); yield report(context, { node, summary: `computed expression statement`, detail: 1 }); }; executors.VariableDeclaration = function* (node, context, { report }) { const kind = node.kind; yield report(context, { node, pre: true, summary: `${kind} variable declaration`, detail: 1 }); for (const declaration of node.declarations) { yield* exec(declaration, context, { report, kind }); } yield report(context, { node, summary: `completed ${kind} variable declaration`, detail: 1 }); }; executors.VariableDeclarator = function* (node, context, { report, kind }) { if (kind !== "let" && kind !== "const") throw new NotSupported(`only let/const supported, not ${kind}`); if (node.id.type !== "Identifier") throw new NotImplemented(`Not yet able to declare non-identifiers :(`); const scope = context.scopes[context.currentScope]; const name = node.id.name; scope.variables[name] = { name, kind }; yield report(context, { node: node.id, summary: `declared ${name}`, detail: 2 }); if (node.init) { const value = yield* exec(node.init, context, { report }); scope.variables[name].value = value; yield report(context, { node, summary: `initialized ${name}`, detail: 2 }); } }; executors.NullLiteral = function* (node, context, { report }) { yield report(context, { node, summary: `evaluated null`, detail: 2 }); return null; }; executors.StringLiteral = function* (node, context, { report }) { yield report(context, { node, summary: `evaluated "${node.value}"`, detail: 2}); return node.value; }; executors.NumericLiteral = function* (node, context, { report }) { yield report(context, { node, summary: `evaluated ${node.value}`, detail: 2 }); return node.value; }; executors.BooleanLiteral = function* (node, context, { report }) { yield report(context, { node, summary: `evaluated ${node.value}`, detail: 2 }); return node.value; }; exec._executors = executors; return exec; }34:T686,{ function displayCat({ title, ks }) { return `

${title} (${ks.length})

${k}

`; } const categories = [ { title: "Won't", check: k => k.match(/^JSX/) || k.match(/^Type/) || k.match(/^Declare/) || k.match(/^Pipeline/) || k.match(/^Export/) || k.match(/^Import/) || k.match(/^ObjectType/) || k.match(/^ClassPrivate/) || k.match(/TypeAnnotation$/) || k.match(/Directive/), ks: ["MetaProperty", "BigIntLiteral", "PrivateName", "Decorator", "Variance", "LabeledStatement", "WithStatement", "InferredPredicate", "InterfaceExtends", "InterfaceDeclaration", "OpaqueType", "QualifiedTypeIdentifier"] }, { title: "Might", check: k => k.match(/^TS/), ks: ["AwaitExpression", "Placeholder", "YieldExpression"] }, { title: "DONE", check: k => Boolean(execute._executors[k]), ks: [] }, { title: "TODO", check: k => true, ks: [] } ]; ALL_NODE_TYPES.forEach(k => { for (const cat of categories) { if (cat.check(k)) { cat.ks.push(k); } if (cat.ks.includes(k)) return; } }); return html`

Status

${categories.map(displayCat).join(``)}

`; }35:[] 2f:{"title":"Visualizing JS execution","cells":[{"id":0,"value":"md`# Visualizing JS execution`","pinned":false,"hidden":false,"mode":"ojs"},{"id":537,"value":"md`***W O R K I N P R O G R E S S***`","pinned":false,"hidden":false,"mode":"ojs"},{"id":370,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":1125,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1226,"value":"md`**Example #1 - Closure**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":947,"value":"{\n const { code, history } = ex;\n return html`\n

${highlight(code, history[stepno].info.node.loc, history[stepno].info.pre ? \"#fffad2\" : \"#d5e6ed\")}

${drawContext(history[stepno].context)}

[step ${stepno + 1} / ${history.length}] ${history[stepno].info.summary}

\n `\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":941,"value":"viewof stepno = slider({ title: \"Execution step\", value: 8, min: 0, max: ex.history.length - 1, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":965,"value":"viewof max_detail = slider({ title: \"Level of detail\", value: 1, min: 1, max: 2, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":934,"value":"ex = {\n const code = `const ans = 42;\nlet i = 0;\ni = i + 1;\nfunction hello() {\n const ans = \"Hello\";\n return person => ans + \", \" + person;\n}\nconst res = hello()(\"Kelley\");`\n \n const history = recordHistory(babel.parse(code).program)\n .filter(({ info }) => info.detail <= max_detail)\n \n return { code, history };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1241,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1280,"value":"md`**Example #2 - Closure**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1252,"value":"{\n const { code, history } = ex3;\n return html`\n

${highlight(code, history[stepno_3].info.node.loc, history[stepno_3].info.pre ? \"#fffad2\" : \"#d5e6ed\")}

${drawContext(history[stepno_3].context)}

[step ${stepno_3 + 1} / ${history.length}] ${history[stepno_3].info.summary}

\n `\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1250,"value":"viewof stepno_3 = slider({ title: \"Execution step\", value: 62, min: 0, max: ex3.history.length - 1, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1248,"value":"viewof max_detail_3 = slider({ title: \"Level of detail\", value: 2, min: 1, max: 2, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1244,"value":"ex3 = {\n const code = `let adj = \"kind\";\nconst f = a => {\n return thing => a + \" \" + adj + \" \" + thing;\n};\nconst definite = f(\"the\");\nconst random = f(\"some\");\nconst s1 = definite(\"variable\");\nadj = \"happy\";\nconst s2 = random(\"interpreter\");\n`\n \n const history = recordHistory(babel.parse(code).program)\n .filter(({ info }) => info.detail <= max_detail_3)\n \n return { code, history };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1174,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1230,"value":"md`**Example #3 - Objects**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1194,"value":"{\n const { code, history } = ex2;\n return html`\n

${highlight(code, history[stepno_2].info.node.loc, history[stepno_2].info.pre ? \"#fffad2\" : \"#d5e6ed\")}

${drawContext(history[stepno_2].context)}

[step ${stepno_2 + 1} / ${history.length}] ${history[stepno_2].info.summary}

\n `\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1190,"value":"viewof stepno_2 = slider({ title: \"Execution step\", value: 0, min: 0, max: ex2.history.length - 1, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1188,"value":"viewof max_detail_2 = slider({ title: \"Level of detail\", value: 2, min: 1, max: 2, step: 1 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1177,"value":"ex2 = {\n const code = `const interpreter = { happy: true };\nconst o = { interpreter };\no.interpreter[\"complete\"] = false;`\n\n const history = recordHistory(babel.parse(code).program)\n .filter(({ info }) => info.detail <= max_detail_2)\n \n return { code, history };\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1131,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1133,"value":"md`\n### Thoughts\n- Some different kinds of scopes (not important for execution, through important for elegant visualization):\n - func function body execution scope (\"the original scope\")\n - let let/const introduced scope (much de-emphasis, maybe just hide)\n - block block statement scope (de-emphasize, but indent)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1154,"value":"html`\n `;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":787,"value":"function highlight(code, loc, color = \"#d5e6ed\") {\n const lines = code.split(\"\\n\");\n\n const before = [\n ...lines.slice(0, loc.start.line - 1),\n lines[loc.start.line - 1].slice(0, loc.start.column)\n ].join(\"\\n\");\n\n const highlighted = loc.start.line === loc.end.line\n ? lines[loc.start.line - 1].slice(loc.start.column, loc.end.column)\n : [\n lines[loc.start.line - 1].slice(loc.start.column),\n ...lines.slice(loc.start.line, loc.end.line - 1),\n lines[loc.end.line - 1].slice(0, loc.end.column)\n ].join(\"\\n\");\n\n const after = [\n lines[loc.end.line - 1].slice(loc.end.column),\n ...lines.slice(loc.end.line)\n ].join(\"\\n\");\n\n return html`${before}${highlighted}${after}`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":460,"value":"deepcopy = require(\"deepcopy\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":5,"value":"babel = require('https://bundle.run/@babel/parser@7.5.0')","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"t = require('https://bundle.run/@babel/types@7.5.0')","pinned":false,"hidden":false,"mode":"ojs"},{"id":1324,"value":"ALL_NODE_TYPES = Object.keys(t)\n .filter(k => k.length > 0 && k[0].toUpperCase() === k[0])\n .filter(k => k.toUpperCase() !== k)","pinned":false,"hidden":false,"mode":"ojs"},{"id":938,"value":"import { slider } from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W35","stdlib":"1","titling":"auto"}