30:T44d,inputSpec = ({ "width": 200, "height": 100, "data": [ { "name": "table", "values": [ {"key": "A", "value": 4}, {"key": "B", "value": 10}, {"key": "C", "value": 7} ] } ], "scales": [ { "name": "xscale", "type": "band", "domain": {"data": "table", "field": "key"}, "padding": {"signal": "scalePadding"}, "range": "width" }, { "name": "yscale", "type": "linear", "domain": {"data": "table", "field": "value"}, "range": "height" } ], "marks": [ { "type": "rect", "from": {"data": "table"}, "encode": { "update": { "x": {"scale": "xscale", "field": "key"}, "y": {"scale": "yscale", "field": "value"}, "y2": {"scale": "yscale", "value": 0}, "width": {"scale": "xscale", "band": 1}, "fill": {"value": "steelblue"} } } } ], "signals": [ { "name": "scalePadding", "value": 0.1, "bind": {"input": "range", "min": 0, "max": 0.99, "step": 0.01} } ] })31:T419,md`

## Visualizations as Reactive Dataflows For a given input specification, Vega generates a _dataflow graph_ that performs the computations needed to map data to visual elements, add axes or legends, and respond to interactive updates. A dataflow graph contains a set of _nodes_ (also called _operators_) that perform operations such as calculating an aggregate or creating a scale mapping. Nodes can process values from other nodes and/or streams of data objects (such as input data sources, derived data sources, or scenegraph items representing visual elements). Graph _edges_ represent dependencies between nodes. Each node may have an associated value. For example, a node corresponding to a Vega signal contains the signal value, whereas a node corresponding to a scale contains a [d3-scale](https://github.com/d3/d3-scale) instance. There are two types of edges: _parameter_ edges indicate that one node directly depends on the value of another node, while _pulse_ edges indicate that data objects "flow" from one node to another. `32:Tc7a,md`Trace the dataflow graph to see how calculations are carried out. For example, given the input dataset \`table\`, an \`extent\` operator calculates the minimum and maximum values of the \`value\` field to determine the \`yscale\` \`domain\` parameter. Similarly, a chain consisting of \`aggregate\`, \`collect\`, and \`values\` operators calculates unique \`key\` field values to determine the \`xscale\` \`domain\`. The \`xscale\` operator is also parameterized by the \`scalePadding\` signal. The \`table\` dataset is encoded as rectangle marks by a sequence of operators: - A \`datajoin\` operator maps input data objects to scenegraph mark items. - A \`collect\` operator gathers the output of the data join. - A \`mark\` operator binds the generated mark items to the scenegraph. - An \`encode\` operator computes visual encodings to determine mark item properties, and is parameterized by \`xscale\` and \`yscale\`. - A \`bound\` operator computes bounding boxes for each mark item. The bounds are used for both layout and input event hit testing. - A \`render\` operator queues scenegraph updates for (re-)rendering. - Finally, a \`sieve\` operator propagates mark updates (but not the mark items themselves) to other parts of the dataflow graph, causing downstream operators to be re-evaluated if and when any mark items change. In this case, the top-level \`viewlayout\` is re-run when mark items change. The other set of visual encoding operators — starting from the \`root\` node — calculate properties and layout for the visualization's top-level group mark. `33:T54e,md`

## View & Dataflow Runtime Once an input specification has been parsed to a dataflow graph description, we can instantiate an interactive [\`View\` component](https://github.com/vega/vega/tree/master/packages/vega-view). The \`View\` takes the parsed specification as input and uses the [vega-runtime](https://github.com/vega/vega/tree/master/packages/vega-runtime) library to turn the dataflow graph description into a live [\`Dataflow\`](https://github.com/vega/vega/tree/master/packages/vega-dataflow) instance. A Vega \`Dataflow\` evaluates nodes in topological order, ensuring that all dependencies are processed before an operator is evaluated. Internally, [\`Pulse\`](https://github.com/vega/vega/blob/master/packages/vega-dataflow/src/Pulse.js) instances are used to pass data from one node to another, tracking which data objects have been inserted, removed, or modified. A major advantage to modeling computation as a dataflow graph is the ability to perform efficient reactive updates. When parameters change or the input data is modified, the dataflow can re-evaluate only those nodes affected by the update. Adjust the \`stage\` drop-down below to see which nodes are re-evaluated when the \`scalePadding\` signal is updated by dragging the bound slider. `34:T659,md`

## Next Steps Knowing how Vega works can be helpful when debugging your own visualizations. The [Vega View API](https://vega.github.io/vega/docs/api/view/) provides access to data sources, scales, and the scenegraph. For example, \`view.scenegraph().root\` accesses the scenegraph tree. The \`view._runtime\` property provides access to the _runtime context_ of dataflow operators. The object \`view._runtime.nodes\` is a map of all dataflow operators in the top-level scope. Some more advanced aspects are not covered in this notebook. In particular, Vega supports _nested scopes_: group marks containing signal, data, and scale definitions within their own namespace. Vega implements this functionality via _subflows_, which can be thought of as recursively-defined dataflow graphs. Moreover, via _faceting_, nested groups can be defined in a data-driven way, requiring Vega to dynamically add and remove subflows in response to streaming data or user input! Nested scopes can be inspected using the \`view._runtime.subcontext\` array. To learn more about the main ideas behind Vega, including declarative specification of interaction techniques, see Arvind Satyanarayan's 2016 OpenVisConf talk ["Reactive Building Blocks: Interactive Visualizations with Vega"](https://www.youtube.com/watch?v=Y8Fp9z-9DWc), or our InfoVis 2015 paper ["Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization"](http://idl.cs.washington.edu/papers/reactive-vega-architecture/). Though those resources describe Vega 2.0 and this notebook describes Vega 3.0+, the central concepts are the same! `35:[] 2e:{"title":"How Vega Works","cells":[{"id":0,"value":"md`# How Vega Works\n\n_A step-by-step guide to how Vega JSON specifications become interactive views. This notebook assumes basic familiarity with Vega, at the level of the [\"Let's Make a Bar Chart\" tutorial](https://vega.github.io/vega/tutorials/bar-chart/)._\n\nThe [Vega visualization grammar](https://vega.github.io/vega) is a [declarative language](https://en.wikipedia.org/wiki/Declarative_programming) for describing visualizations. The Vega runtime parses specifications in a JSON format to produce interactive web-based graphics. One unique aspect of Vega is its support for declarative interaction using a reactive model: as input events arrive and data or variables change, Vega efficiently updates the visualization.\n\nLet's start with a simple bar chart that includes a basic form of interaction: a slider that adjusts the amount of padding added by the x-axis scale encoding.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":182,"value":"viewof view = {\n const div = document.createElement('div');\n div.value = new vega.View(parsedSpec)\n .initialize(div)\n .run();\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":492,"value":"md`Expand the \\`inputSpec\\` cell below to see the Vega JSON specification for the chart.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":105,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":254,"value":"md`The specification consists of an input data set, two scale transforms (\\`xscale\\` and \\`yscale\\`), and a \\`rect\\` mark definition that uses the scales to visually encode the data as rectangles.\nIn addition, a \\`scalePadding\\` signal is defined to parameterize the padding of \\`xscale\\`, along with a binding to a slider input widget.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":212,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":608,"value":"md`The diagram below visualizes the dataflow graph for our bar chart.\nGray nodes indicate signals,\nred nodes indicate data sources,\ngreen nodes indicate scales, and\nblue nodes process visualized marks.\nWhite nodes are internal data operations that calculate needed statistics or propagate updates.\n\nBlack edges indicate **pulse** dependencies, gray edges are parameter dependencies.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":192,"value":"dot`${view2dot(viewHide)}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":616,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":624,"value":"md`

\n## Parsing a Vega Specification\n\nThe first step in going from an input specification to a live dataflow is _parsing_. The [vega-parser](https://github.com/vega/vega/tree/master/packages/vega-parser) module takes a declarative JSON specification as input and produces a _dataflow graph description_ as output. This output is only a description: think of it as a file format for a dataflow graph.\n\nExpand the \\`parsedSpec\\` object below to inspect the parser output for our \\`inputSpec\\`.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":229,"value":"parsedSpec = vega.parse(inputSpec)","pinned":false,"hidden":false,"mode":"ojs"},{"id":235,"value":"md`\nThe \\`operators\\` property is a list of dataflow nodes. Dependencies are indicated via references in a node entry's \\`params\\` object: \\`operators[11].params.padding: {$ref: 6}\\` indicates that the \\`yscale\\` (node \\`12\\`) depends on the value of the \\`scalePadding\\` signal (node \\`6\\`). The \\`bindings\\` property describes input bindings, here between the \\`scalePadding\\` signal and a slider.\n\nOther properties of the output — unused in this example — include \\`streams\\` (event stream definitions such as mouse move, click, _etc._), \\`updates\\` (expressions for updating nodes in response to events), and the overall view \\`background\\` color.\n\nThe parser compiles encoding logic and expressions to JavaScript code. For example, the \\`update\\` encoding block in the input specification maps to this JavaScript code in \\`operators[22].params.encoders.$encode.update\\`:\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":708,"value":"{\n const c = parsedSpec.operators[22].params.encoders.$encode.update.$expr.channels;\n let s = ' ';\n for (const key in c) {\n s += `o.${key} = ${c[key].code};\\n `;\n }\n return md`~~~ js\\n${s}~~~`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":256,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":328,"value":"viewof stage = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":316,"value":"dot`${view2dot(viewHide, +stage)}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":338,"value":"md`

\n## Scenegraph & Rendering\n\nThe Vega \\`Dataflow\\` constructs and maintains a [scenegraph](https://en.wikipedia.org/wiki/Scene_graph): a tree data structure that models all the visual elements, including the position, color, shape, and size of visual marks. The [vega-scenegraph](https://github.com/vega/vega/tree/master/packages/vega-scenegraph) module traverses the scenegraph to perform _rendering_, drawing the view using web standards such as [Canvas](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API) and [SVG](https://developer.mozilla.org/en-US/docs/Web/SVG).\n\nExpand the \\`scene\\` object below to inspect the scenegraph for our bar chart.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":345,"value":"scene = JSON.parse(view.scenegraph().toJSON())","pinned":false,"hidden":false,"mode":"ojs"},{"id":508,"value":"dot`${scene2dot(viewHide)}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":342,"value":"md`The levels of the tree alternate between enclosing _marks_ and constituent mark _items_. The first two levels are the \\`root\\` group mark and its single group item, which contains the visualization. Expanding down to \\`scene.items[0].items[0]\\`, we find a \\`rect\\` mark containing three items: one for each bar in the bar chart.\n\nThe \\`render\\`\ndataflow operator informs the scenegraph of items that have been modified. In response, the renderers also perform efficient incremental updates, either by redrawing only the affected pixels in a Canvas (damage/redraw) or modifying only the relevant SVG elements.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":240,"value":"md`

\n## Adding An Axis\n\nThe discussion above focuses on a simple example in which the graphical output is only three rectangles. Let's consider a more realistic chart by adding a y-axis to our original input specification:\n\n\\`\\`\\`json\n${JSON.stringify(axisDef, 0, 2)}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":246,"value":"viewof view2 = {\n const div = document.createElement('div');\n div.value = new vega.View(vega.parse(vega.extend({}, inputSpec, +axisSelector ? axisDef : null)))\n .initialize(div)\n .run();\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":129,"value":"viewof axisSelector = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":416,"value":"md`Adjust the axis selector drop-down above to switch between our original bar chart and a new one with a y-axis. Note below how much more complex the dataflow graph is when the axis is included! [Creating appropriate axes and legends is a critical, yet surprisingly involved, part of visualization design.](https://beta.observablehq.com/@vega/a-guide-to-guides-axes-legends-in-vega)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":418,"value":"dot`${view2dot(view2)}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":434,"value":"md`Internally, the Vega parser processes _axis_ definitions to generate an axis group, an\n\\`axisticks\\` data source\n(parameterized by \\`yscale\\`),\nand visual encoding operators for ticks, labels, and gridlines. Vega uses a similar approach to generate legends.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":420,"value":"scene2 = JSON.parse(view2.scenegraph().toJSON())","pinned":false,"hidden":false,"mode":"ojs"},{"id":511,"value":"dot`${scene2dot(view2)}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":446,"value":"md`When the y-axis is included, the \\`scene2\\` scenegraph object includes a group mark containing the axis items under \\`scene2.items[0].items[0]\\`.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":480,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":404,"value":"md`

\n## Imports & Extras`","pinned":false,"hidden":false,"mode":"ojs"},{"id":401,"value":"axisDef = ({\n \"axes\": [\n {\n \"orient\": \"left\",\n \"scale\": \"yscale\",\n \"grid\": true,\n \"tickCount\": 5\n }\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":320,"value":"// headless view to use so that other cells don't update in\n// response to signal updates on the initial view cell\nviewHide = {\n const view = new vega.View(parsedSpec).initialize();\n view.runAsync().then(function(view) {\n view.signal('scalePadding', 0.5).run();\n });\n return view;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":7,"value":"vega = require('https://cdn.jsdelivr.net/npm/vega@5/build/vega.js')","pinned":false,"hidden":false,"mode":"ojs"},{"id":32,"value":"dot = require(\"@observablehq/graphviz@0.2.1\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":174,"value":"import {scene2dot, view2dot} from '@vega/vega2dot'","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W35","stdlib":"1","titling":"auto"}