31:T462,Over the past two years during my PhD at Berkeley, I've been developing a new direct manipulation programming system for geospatial analysis and visualization, [`cartokit`](https://github.com/parkerziegler/cartokit). Selfishly, I think it's pretty neat! At a high level, `cartokit` allows users to load, transform, and style geospatial data using familiar interactions from direct manipulation interfaces (e.g., [Figma](https://www.figma.com/), [Adobe Illustrator](https://www.adobe.com/products/illustrator.html)) while simultaneously generating JavaScript programs that produce a corresponding interactive map. The system’s design was heavily influenced by my [prior observational research](https://dl.acm.org/doi/10.1145/3544548.3581370) in the data journalism, Earth and climate science, and social science communities, in which I found that many domain expert mapmakers are extremely proficient GUI users but struggle to translate static map designs to code. `cartokit` is an attempt to fill that gap, marrying the best of direct manipulation with the rigorous correctness guarantees of a formal programming system.32:T45c,Since `cartokit`'s earliest versions, I have only targeted JavaScript paired with Mapbox GL JS as the system's primary backend. However, the ecosystem of languages and libraries available for interactive mapping today is incredibly rich, and the choice to use one pairing over another—or even switch between them—is highly context-dependent. [MapLibre GL JS ](https://maplibre.org/maplibre-gl-js/docs/) is an increasingly popular alternative with a similar WebGL2 API and first-class support for a new, efficient tile format for geospatial data, [PMTiles](https://docs.protomaps.com/pmtiles/). [deck.gl](https://deck.gl/) has best-in-class rendering performance and is a popular choice for geospatial datasets in the 1+ million feature range. [D3](https://d3js.org/) and [Leaflet](https://leafletjs.com/) are older standbys, but have been the de facto standard in newsrooms for nearly a decade. Extending beyond the JavaScript ecosystem, the Python and R ecosystems have their own bindings to these renderers to facilitate interactive cartography from languages with first-class geospatial analysis capabilities.33:Ta9f,So that's all well and good, but what does multi-backend support entail concretely? For the scope of this work, I decided to focus on two primary objectives. 1. **Extending `cartokit`'s code generator to support MapLibre GL JS as a library target and TypeScript as a language target.** Accomplishing this objective would give users a total of four possible backends:

Mapbox GL JS with JavaScript
Mapbox GL JS with TypeScript
MapLibre GL JS with JavaScript
MapLibre GL JS with TypeScript

It may not seem like much, but this is a 4${tex`{\times}`} growth in the space of possible platforms `cartokit` can support. Anecdotally, I've seen all four in use in production in my time in industry and the newsroom.

2. **Supporting dynamic, on-demand loading of compiler backends.** With more backends comes more code, and with more code comes larger bundles that need to be fetched, parsed, and executed by the browser's JavaScript engine (e.g., V8, JavaScriptCore, etc.). Large bundles end up being a problem for several reasons:

Network speed. If a user is on a slow 3G or 4G connection—or even if they're using UC Berkeley's WiFi on a bad day—a larger bundle increases the time it takes for all of your code from your web server to reach the user's browser.
JavaScript parse time. Once your code has been transferred to the client, the JavaScript engine needs to parse it! Thankfully, parsing JavaScript in modern engines is ["blazingly" fast](https://www.reddit.com/r/rustjerk/comments/sqy221/anything_built_in_rust_is_blazingly_fast_by/). Still, more code to parse means longer parse times.
JavaScript execution time. We've fetched the code, we've parsed it, and now we need to run it! Again, it's the same story here—more code to execute amounts to more time before the main content of our interface loads.

In short, adding more backends could have compounding negative impacts on application performance if we're not careful. Thankfully, modern JavaScript build tools like [Rollup](https://rollupjs.org/) have a great solution for this problem: [code splitting](https://rollupjs.org/tutorial/#code-splitting). In essence, code splitting allows you to dice your application code into separate chunks that can loaded efficiently (e.g., via HTTP/2 multiplexing) or "just-in-time" (i.e., dynamically) using JavaScript's [dynamic `import` statement](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import). Code splitting is going to play a big role in making multi-backend support feasible from a performance perspective.

34:T48b,One observation about this structure is that the `codegen*` functions are skewed more closely towards the API structure of the compilation target (i.e., Mapbox GL JS) than the source program (i.e., the `cartokit` DSL program). For example, the `cartokit` DSL has no notion of a data source, but our code generator has a dedicated function (`codegenSource`) that produces JavaScript code for a [Mapbox GL JS source](https://docs.mapbox.com/mapbox-gl-js/api/sources/). I think this is largely a relic of the fact that, when I started out, I was really only thinking about supporting one backend. At the time, it just made sense to me to skew code generation more closely to that target. However, an alternative design could just as easily skew towards the DSL. For example, I could have captured the creation of a data source and a layer together in a single `codegenLayer` function and never created `codegenSource`. This would more closely mimic the design of `cartokit`'s DSL, which has its own layer abstraction (`CartoKitLayer`) that stores a map layer's geospatial data alongside its stylistic definition. Alas, choices in language design—can't escape them!35:T45b,Moreover, when it comes to handling library discrepancies in code generation, full separation of backends is almost required from both a maintenance and performance perspective. While Mapbox GL JS and MapLibre GL JS have (intetionally) very similar APIs, they do diverge significantly in areas where I'm targeting future work (e.g., Mapbox GL JS has no support for MapLibre GL JS's [`addProtocol` API](https://maplibre.org/maplibre-gl-js/docs/API/functions/addProtocol/) and MapLibre GL JS does not yet support [custom projections](https://docs.mapbox.com/mapbox-gl-js/guides/projections/)). Moreover, while prototyping [deck.gl](https://deck.gl/) code generation as part of this project, I've found that the code generation template boundaries (e.g., layers, sources, paint, transformations, etc.) that I use for Mapbox GL JS and MapLibre GL JS feel stretched by deck.gl's APIs. For example, deck.gl's definition of layers—which colocate data and style properties—actually matches the `cartokit` DSL much more closely. For this backend target, a `codegenSource` function just doesn't make a whole lot of sense.36:T82e,While I embraced the "separate-all-the-things" mentality when it came to code generation, I did find one area where generalization made more sense—analysis of the `cartokit` DSL program. `cartokit`'s analysis phase is critical for code generation and handles determining: 1. **Requirements around the program's library and file imports.** For example, if a user triggers an interface action that requires geometric transformation of the underlying GeoJSON data (e.g., changing a `Choropleth` layer to a `Proportional Symbol` layer), `cartokit` needs to `import` specific functions of the [Turf.js library](https://turfjs.org/) (e.g., [`centroid`](https://turfjs.org/docs/api/centroid)) to encode this transformation into the program. 2. **Requirements around data fetching and asynchronous execution.** For example, if a user supplies an API endpoint to a GeoJSON file as a data source, the generated `map.on('load')` callback can still be marked as a synchronous function using [Mapbox GL JS's and MapLibre GL JS's support for urls as valid values for a GeoJSON `source`'s `data` property](https://docs.mapbox.com/style-spec/reference/sources/#geojson-data). However, if a user triggers an interface action to transform this data in any way, `cartokit` must fetch the data from the API endpoint and store the result in memory for later local transformation. This involves inserting an asynchronous function, `fetchGeoJSON`, into the program's top-level scope, inserting a call site for this function inside the `map.on('load')` callback and, finally, marking the callback as asynchronous using the `async` keyword. 3. **Requirements around transformation composition.** For example, if a user applies both a geometric transformation and a separate tabular transformation to a layer's GeoJSON data, `cartokit` needs to compose those transformations in the correct order in the generated program. We do this by `import`ing `lodash`'s [`flow`](https://lodash.com/docs/4.17.15#flow) function and determining the proper scope for inserting a call to `flow` that composes the two transformations.37:T51b,| ID | Map | Newsroom | LOC (Max) | | --- | ------ | -------- | --- | | 1 | ["Maps of the April 2024 Total Solar Eclipse"](https://www.nytimes.com/interactive/2024/science/total-solar-eclipse-maps-path.html) | The New York Times | 56 | | 2 | ["You’re not crazy. Spring is getting earlier. Find out how it’s changed in your town."](https://www.washingtonpost.com/climate-environment/interactive/2024/spring-earlier-arrival-plants-map/) | The Washington Post | 42 | | 3 | ["Winter is warming almost everywhere. See how it’s changed in your town."](https://www.washingtonpost.com/climate-environment/interactive/2024/winter-temperature-warming-city-data-climate-change/) | The Washington Post | 40 | | 4 | ["A boat went dark. Finding it could help save the world's fish."](https://www.washingtonpost.com/climate-environment/interactive/2023/map-illegal-fishing/) | The Washington Post | 47 | | 5 | ["Bird populations are declining. Some are in your neighborhood."](https://www.washingtonpost.com/climate-environment/interactive/2024/bird-population-decline-united-states-maps/) | The Washington Post | 60 | | 6 | ["Will global warming make temperature less deadly?"](https://www.washingtonpost.com/climate-environment/interactive/2023/hot-cold-extreme-temperature-deaths/) | The Washington Post | 47 |38:T51d,To start off, I worked to to evaluate the correctness of generated programs for each backend. Correctness always feels like a fuzzy notion to me in programming languages. Correct with respect to what? A formal specification? Some imaginary notion of user intent? It's a can of worms, but for the purposes of this evaluation I used the following criteria: 1. Syntactic and semantic validity of generated programs with respect to the language-library combination of the backend 2. “Perceptual” equivalence to outputs yielded by other backends, as well as the ground truth map I evaluated the former metric by executing the program generated by each backend in a newly scaffolded Vite project and ensuring (1) syntactic validity, (2) absence of run time errors, and (3) absence of type errors for programs generated by the TypeScript backends. I evaluated the latter metric by a qualitative assessment, comparing the generated maps to each other and the original benchmark. I deemed two maps equivalent if they demonstrated the same visual symbology modulo small rendering discrepancies between libraries. Figure 4 shows one example of what this evaluation looked like, with the original map from The Washington Post, the map developed in `cartokit`, and the four output maps produced by the four backends.39:T4fb,benchmarkPlots = { const data = _.groupBy(codegen, "playwrightWorkflowId"); const orderedData = Object.keys(data) .sort() .reduce((obj, key) => { obj[key] = data[key]; return obj; }, {}); return htl.html`${Object.entries(orderedData).map(([key, entries]) => { return Plot.plot({ height: 200, title: `Benchmark ${key.split("-")[1]}`, x: { axis: "top", label: "Duration (ms)", grid: true }, fy: { grid: true, tickFormat: (d) => d .replace("maplibre", "MapLibre GL JS") .replace("mapbox", "Mapbox GL JS") .replace("javascript", "JavaScript") .replace("typescript", "TypeScript") .replace("-", " / "), label: "Backend", padding: 0 }, facet: { data: entries, y: "backend", marginLeft: 180 }, marks: [ Plot.dot( entries, Plot.dodgeY( { anchor: "middle" }, { x: "duration", fill: "backend", fillOpacity: 0.25, stroke: "backend", r: 3 } ) ) ] }); })}`; }3a:T5a8,Examining the sum size of all JavaScript modules loaded on initial render in a production build, the multi-backend version of `cartokit` loads **499.1kb** while the single-backend version of `cartokit` loads **665.25kb**. **Thus, multi-backend support actually _decreases_ our initial bundle size by ≈25%.** This result feels like a resounding success for code splitting. While it may seem paradoxical that we added _more code_ (4${tex`{\times}`} the number of backends) and yet have a _smaller initial bundle_, recall how Rollup creates chunks while building `cartokit`. Previously, in our single-backend architecture, our backend was statically (not dynamically) `import`ed. In this setting, Rollup had no way to infer from the module graph alone that our backend would not be needed for the initial page load and, thus, opted to include it in the main chunk. In our multi-backend architecture, all backends are dynamically `import`ed. The use of dynamic `import` acts almost like a chunking hint to Rollup here, indicating that a given backend doesn't need to be loaded until requested. Given that the generated program in `cartokit` is not shown to the user unless they choose to view it by clicking the **View Editor** button, it's actually safe to load _none of the backends_ on initial load. This helps to explain why we actually observe a in initial bundle size rather than a stable value in the switch to the multi-backend architecture.3b:T4c7,The median LCP across the runs was **${d3.median(lcp.filter(d => d.condition === 'single'), d => d.lcp)}s for the single-backend version** and **${d3.median(lcp.filter(d => d.condition === 'multiple'), d => d.lcp)}s for the multi-backend version**—again, nearly identical. This one was actually somewhat surprising for me. Given that we shaved off roughly 165kb of our bundle, I thought we might see a drop in LCP values. But, I think there are two things at play here: 1. `cartokit` includes a lot of application code for the user interface, and it's this code that needs to execute to actually render the map (and, thus, has the impact on LCP). We didn't remove any of this code using code splitting. 2. `cartokit`'s bundles are still relatively small. A 400-600kb gzipped bundle is pretty small as far as web applications go these days. And browsers—well yeah, they're actually, truly blazingly fast. So, a bit of a womp womp. If I still worked in industry, maybe I could list this 150kb reduction on my KPIs for the quarter. In this instance, it's just a footnote at the long end of this increasingly unhinged project report. Figure 5 shows a strip plot of each LCP values from our ten runs across each condition.3c:T456,If you've made it this far in this write up, you are either (1) Professor Max Willsey or (2) someone who could probably be spending their time more wisely elsewhere, but who I'm ever so grateful to to have in my little corner of nerdery. Ultimately, I'd call this whole venture a success. Code splitting worked as advertised, the "separate-all-the-things" approach to new backends seems like it has some great runway for the near term, and deck.gl support is already in the works. I'm very excited to roll out this iteration of multi-backend support in v0.7.0 in `cartokit`, releasing as soon as I finish up some pressing work in other role as a data journalist in the newsroom at [Grist](https://grist.org/). For the sickos, MapLibre support was added in [this PR](https://github.com/parkerziegler/cartokit/pull/580) and TypeScript support is [up on a branch](https://github.com/parkerziegler/cartokit/tree/feat/add-typescript-codegen). If you liked this notebook, let me know by hitting me up on Bluesky, [`@parkie-doo.sh`](https://bsky.app/profile/parkie-doo.sh) 🦋 Until such time, happy mapping 🗺️3d:[{"id":"20e3a33fb71741e6162d8c08b157ee504130bb2542a64f11748d62a9af3192b5af26a6ef133444d6c9b9ba4aacadc8a107b5cf5399c9346aa0315485895b94da","href":"https://static.observableusercontent.com/files/20e3a33fb71741e6162d8c08b157ee504130bb2542a64f11748d62a9af3192b5af26a6ef133444d6c9b9ba4aacadc8a107b5cf5399c9346aa0315485895b94da","name":"cartokit-benchmark-example.png","type":"image/png","lastModified":1734390458437.325,"size":5229272},{"id":"7d4e22094e956acda98d06a36eaddecacdd41a76b2d8f39ead1abbce135c0a0d7f0574e21b6eeb4c854762cb6615d205d21203062323b9569b23fb60f76cc45e","href":"https://static.observableusercontent.com/files/7d4e22094e956acda98d06a36eaddecacdd41a76b2d8f39ead1abbce135c0a0d7f0574e21b6eeb4c854762cb6615d205d21203062323b9569b23fb60f76cc45e","name":"cartokit-channels-min.mp4","type":"video/mp4","lastModified":1734042970216.498,"size":5103054},{"id":"f57bb3b216ee9321f4b9d92b2a6af95c4857904a45ba29c6b69feeecc3a52da375f7c4749e3c4650729a47fbfefe107853fe8898a9b9ea5030bb366fc5e780ea","href":"https://static.observableusercontent.com/files/f57bb3b216ee9321f4b9d92b2a6af95c4857904a45ba29c6b69feeecc3a52da375f7c4749e3c4650729a47fbfefe107853fe8898a9b9ea5030bb366fc5e780ea","name":"codegen.json","type":"application/json","lastModified":1734469429103.754,"size":35078},{"id":"4d0484f979fa81d15776da82fc5b5437f1dca10dad9497377eb7ee9a86a129795f0f37dda5710a1299f091fc14329d37ae41272dfc61e0488e00a675681899f2","href":"https://static.observableusercontent.com/files/4d0484f979fa81d15776da82fc5b5437f1dca10dad9497377eb7ee9a86a129795f0f37dda5710a1299f091fc14329d37ae41272dfc61e0488e00a675681899f2","name":"codegen-hierarchy.png","type":"image/png","lastModified":1734214125784.788,"size":60101},{"id":"5e7f8285c170bbdee764f656ad25ee813286f9899da29acf41b776c86fbc5b77432551304f6b100e9b82be05098727420f4c93821ca1f0a1ce36b6ea7ff5ae98","href":"https://static.observableusercontent.com/files/5e7f8285c170bbdee764f656ad25ee813286f9899da29acf41b776c86fbc5b77432551304f6b100e9b82be05098727420f4c93821ca1f0a1ce36b6ea7ff5ae98","name":"cartokit-backend-metrics.json","type":"application/json","lastModified":1734306257407.697,"size":2402},{"id":"8fad7cfa4f1570359401a1aea3ff98ededee2d2784a8cc1f461c951a9ae09ff149e52ceb2ef565d2f10d5b84bb1e0a5ecf3508b7a63344dbb0a0144d63632387","href":"https://static.observableusercontent.com/files/8fad7cfa4f1570359401a1aea3ff98ededee2d2784a8cc1f461c951a9ae09ff149e52ceb2ef565d2f10d5b84bb1e0a5ecf3508b7a63344dbb0a0144d63632387","name":"code-splitting-min.mp4","type":"video/mp4","lastModified":1734334118492.203,"size":2259118}] 2f:{"title":"Multi-backend support in the cartokit compiler","cells":[{"id":0,"value":"# Multi-backend support in the `cartokit` compiler","pinned":false,"hidden":false,"mode":"md"},{"id":5,"value":"_Note: This notebook is part of my final project submission for CS265: Compiler Optimization and Code Generation at UC Berkeley. For that reason, it is slightly more academic-flavored ([academish?](https://www.todepond.com/wikiblogarden/academia/from/the-outside/)) than my other notebooks._","pinned":false,"hidden":false,"mode":"md"},{"id":7,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":29,"value":"Internally, `cartokit` is structured a lot like a traditional compiler, translating programs written in a high-level, JavaScript-embedded domain-specific language (DSL) into plain JavaScript programs using [Mapbox GL JS](https://docs.mapbox.com/mapbox-gl-js/guides). Users modify DSL programs through interface interactions, which produce small syntactic diffs (${tex`\\delta`}) that are applied to the current program in a `patch` procedure. On every call to `patch`, the compiler's backend performs a lightweight analysis of the updated DSL program and generates a corresponding updated JavaScript program. But that's all the under-the-hood stuff—here's what that process looks like in practice from a user's perspective.","pinned":false,"hidden":false,"mode":"md"},{"id":36,"value":"

","pinned":false,"hidden":false,"mode":"html"},{"id":57,"value":"

Figure 1. A user creates a map of powerplants in the United States, using the size of each circle to show the powerplant's capacity and the color of each circle to denote its primary energy source. Notice that cartokit produces an updated JavaScript program on every user interaction in the interface's Properties Panel.

","pinned":false,"hidden":false,"mode":"html"},{"id":34,"value":"$32","pinned":false,"hidden":false,"mode":"md"},{"id":76,"value":"Given this abundant landscape of tooling, I've long wanted to implement \"multi-backend\" support in `cartokit`—in essence, code generation for arbitrary combinations of language-library targets. In addition to being a feature several `cartokit` users in the data journalism community have requested, it also aligns with my deeper dreams of making `cartokit`'s DSL the [LLVM](https://llvm.org/) of the mapping world. And as a research venture, implementing multiple backends is a great way to assess whether my DSL is expressive enough to support diverse language and API constraints while enabling backend-specific optimization.","pinned":false,"hidden":false,"mode":"md"},{"id":93,"value":"## Objectives","pinned":false,"hidden":false,"mode":"md"},{"id":98,"value":"$33","pinned":false,"hidden":false,"mode":"md"},{"id":13,"value":"## Implementation","pinned":false,"hidden":false,"mode":"md"},{"id":151,"value":"With our objectives in place, let's dive into the implementation. We'll discuss the system's previous single-backend architecture, the evolution of this architecture to support multiple backends, and how I added support for dynamic, on-demand loading of backends.","pinned":false,"hidden":false,"mode":"md"},{"id":155,"value":"### The Single-Backend Architecture","pinned":false,"hidden":false,"mode":"md"},{"id":157,"value":"`cartokit`'s backend through v0.5.3 was structured as a hierarchy of small code generation functions (hereafter, `codegen*` functions) that each peek at specific portions of the `cartokit` DSL program to generate a JavaScript program fragment (i.e., a portion of the final output program). In this setup, functions at a higher level in the hierarchy determine where program fragments generated by callees are inserted into the generated program. All-in-all, pretty straightforward! The figure below provides a representation of the call graph for the code generation algorithm.","pinned":false,"hidden":false,"mode":"md"},{"id":163,"value":"codegenHierarchy = FileAttachment(\"codegen-hierarchy.png\").image({ width: 640 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":166,"value":"

Figure 2. The call graph of cartokit's code generation algorithm. Each function in the graph generates its own program fragment, which is placed by its caller into the proper location in the output JavaScript or TypeScript program.","pinned":false,"hidden":false,"mode":"md"},{"id":174,"value":"$34","pinned":false,"hidden":false,"mode":"md"},{"id":222,"value":"The relevant portions of `cartokit`'s original backend can be found [here](https://github.com/parkerziegler/cartokit/tree/v0.5.3/src/lib/codegen).","pinned":false,"hidden":false,"mode":"md"},{"id":177,"value":"### Evolving the Implementation to Support Multiple Backends","pinned":false,"hidden":false,"mode":"md"},{"id":185,"value":"While evolving `cartokit` to support a new library (MapLibre GL JS) and language (TypeScript) backend, I actually decided to reuse much of this same structure. Practically, this meant separating each backend into its own collection of JavaScript modules, each with a _single_ entry point (this will be important later). In source, this amounted to creating a separate directory of code generation functions for every language-library combination (e.g., `lib/codegen/mapbox/javascript/codegen*`, `lib/codegen/maplibre/typescript/codegen*`, etc.). While the given hierarchy of code generation functions (the one shown in Figure 2) is consistent across all backends, this is not a strict requirement of the approach—we could arbitrarily add, remove, or restructure functions independently for a new backend as needed.","pinned":false,"hidden":false,"mode":"md"},{"id":190,"value":"From one perspective, this sounds like a maintenance nightmare—every new addition of a language or library target requires adding _a lot_ of new code. Moreover, for certain targets where elements of code generation are basically identical (e.g., `fill-*` or `stroke-*` expressions in Mapbox GL JS and MapLibre GL JS), we now have multiple versions of every `codegen*` function floating around in source. This is an even more pronounced problem for the addition of the TypeScript backends, which—owing to TypeScript's pretty darn good type inference—can be fully identical to their JavaScript equivalents in many cases.\n","pinned":false,"hidden":false,"mode":"md"},{"id":203,"value":"But, from another perspective, this architecture also allows for extreme specialization of generated code that does not require run time computation. For example, my first attempt at adding TypeScript code generation involved just passing a Boolean flag, `typed`, to all of the JavaScript `codegen*` functions. Then, at run time, each function could check the value of the `typed` flag to determine whether and where it needed to insert static type annotations (e.g., `geojson: FeatureCollection`) or type assertions (e.g., `await response.json() as FeatureCollection`) in its template. Something like this:","pinned":false,"hidden":false,"mode":"md"},{"id":671,"value":"Editor({\n language: \"typescript\",\n value:\n \"function codegenTransformation(name: string, defintion: string) {\" +\n \"\\n\" +\n \"\\treturn `function ${name}(geojson${typed ? ': FeatureCollection' : ''})${typed ? ': FeatureCollection' : '' } {\" +\n \"\\n\" +\n \"\\t\\t${definition}\" +\n \"\\n\" +\n \"\\t}`;\" +\n \"\\n\" +\n \"}\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":669,"value":"In practice, I found that this approach quickly became unwieldly. Nearly every template became littered with conditionals checking the `typed` flag, which made them more difficult to read, check, and extend to new features. Moreover, while evaluating a conditional has almost no run time overhead, it is still more overhead than having no conditionals at all!","pinned":false,"hidden":false,"mode":"md"},{"id":229,"value":"$35","pinned":false,"hidden":false,"mode":"md"},{"id":213,"value":"#### Generalizing Program Analysis","pinned":false,"hidden":false,"mode":"md"},{"id":244,"value":"$36","pinned":false,"hidden":false,"mode":"md"},{"id":280,"value":"Previously, in the single-backend architecture, analysis of a `cartokit` DSL program was intermixed with code generation and ran on demand when a particular `codegen*` function needed information for program fragment production. Now, analysis runs in a separate pass _before_ code generation, mimicking the structure of a more traditional compiler. The information returned from analysis is an `Object` whose values are Boolean flags indicating whether certain libraries and functions are required; many of these flags are also used as proxies by `codegen*` functions to insert or elide parts of their program fragments. Every backend takes the analysis `Object`, in addition to the `cartokit` DSL program, as input.","pinned":false,"hidden":false,"mode":"md"},{"id":282,"value":"### Code Splitting and Dynamic, On-Demand Loading of Backends","pinned":false,"hidden":false,"mode":"md"},{"id":299,"value":"Earlier, we discussed the negative performance implications of larger bundle sizes. With our multi-backend architecture—in which each language-library pairing has a fully isolated set of code generation modules—it may seem that we've set ourselves up for a big performance hit. We now have to load four backends instead of just one. But recall the solution we discussed before: code splitting. If we can effectively split each backend into its own separate chunk, then only one backend will be loaded (i.e., fetched from the web server, parsed, and run over the `cartokit` DSL program) at a given time. This should give us tantamount performance to our prior single-backend architecture.","pinned":false,"hidden":false,"mode":"md"},{"id":323,"value":"The key to making this all work is to pair our build system's support for code splitting with JavaScript's dynamic `import`. `cartokit` uses [Vite](https://vite.dev/) as its build tool, which is backed by [Rollup](https://rollupjs.org/) under the hood. Rollup automatically code splits modules that are `import`ed dynamically, placing these modules into their own \"chunk\". In order to determine the full set of additional modules that should be included in this chunk, Rollup builds and analyzes a module dependency graph. For our purposes, we can exploit this behavior by:\n\n1. Creating a single module entry point for each backend (remember when I said this was import?)\n2. Dynamically `import`ing just that module when a new backend is requested by the user\n\nFrom a code perspective, this is actually pretty lightweight; it looks something like the following:","pinned":false,"hidden":false,"mode":"md"},{"id":341,"value":"viewof codeSplitting = Editor({\n language: \"typescript\",\n value:\n \"function loadBackend(dslProgram, libraryBackend, languageBackend) {\" +\n \"\\n\" +\n \"\\tconst module = import(`\" +\n \"\\n\" +\n \"\\t\\tlib/codegen/${libraryBackend}/${languageBackend}/index.js\" +\n \"\\n\" +\n \"\\t`)\" +\n \"\\n\\n\" +\n \"\\tmodule\" +\n \"\\n\" +\n \"\\t\\t.then(({ codegen }) => codegen(dslProgram))\\n\" +\n \"\\t\\t.then((program) => {\\n\" +\n \"\\t\\t\\tconsole.log('Output program: ', program);\\n\" +\n \"\\t\\t});\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":353,"value":"If we actually pop open the Network tab in the browser on `cartokit`, we can see this in action! Notice that as I change the choice of backend using the radio buttons in the bottom right corner of the interface, the code generation modules for _just_ the selected language-library backend are requested from the server.","pinned":false,"hidden":false,"mode":"md"},{"id":358,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":366,"value":"

Figure 3. Dynamically loading new backends in cartokit using code splitting. Notice that when a user changes the choice of backend by altering the Language or Library radio buttons, the code generation modules comprising that backend are requested from the web server. These modules are fetched, parsed, and executed to produce the updated program shown to the user.","pinned":false,"hidden":false,"mode":"html"},{"id":706,"value":"Et voila! With a few small changes and some big help from modern JavaScript build tooling, we're now loading new backends on-demand when a user requests them rather than loading them all up front when the page first loads. Beautiful!","pinned":false,"hidden":false,"mode":"md"},{"id":15,"value":"## Evaluation","pinned":false,"hidden":false,"mode":"md"},{"id":371,"value":"With the implementation finished off, my last step was to evaluate my multi-backend architecture. The implementation gave me a hunch that I had successfully enabled this feature without sacrificing correctness of the generated code or system performance. But, until you measure it, you never really know!","pinned":false,"hidden":false,"mode":"md"},{"id":381,"value":"### Benchmark Suite","pinned":false,"hidden":false,"mode":"md"},{"id":383,"value":"As part of my evaluation (and some of my prior research), I developed a set of six benchmarks based on maps published by two national newsrooms: The New York Times and The Washington Post. Using the original data published by these newsrooms, I reproduced each target map in `cartokit` and recorded the program generated by each of the four backends. The table includes information on each benchmark. LOC (Max) indicates the maximum number of lines of code generated by any of the four `cartokit` backends to produce the final map. In all cases, the TypeScript / Mapbox GL JS backend generated the largest number of LOC.","pinned":false,"hidden":false,"mode":"md"},{"id":406,"value":"$37","pinned":false,"hidden":false,"mode":"md"},{"id":373,"value":"### Correctness","pinned":false,"hidden":false,"mode":"md"},{"id":375,"value":"$38","pinned":false,"hidden":false,"mode":"md"},{"id":393,"value":"cartokitBenchmarkExample = FileAttachment(\n \"cartokit-benchmark-example.png\"\n).image({ width: 800 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":397,"value":"

Figure 4. An example benchmark from my evaluation. For this benchmark, I'm using a map from journalist Harry Stevens published in his piece , \"Bird populations are declining. Some are in your neighborhood.\" published in the Washington Post on January 17, 2024. Beneath the Original Map, we can see what its replication looks like in the cartokit interface. After extracting the generated program from cartokit for each language-library backend, I executed the programs in a new Vite project to produce the maps on the bottom row. In this case, the output is pixel-for-pixel equivalent across all language-library backends.","pinned":false,"hidden":false,"mode":"md"},{"id":430,"value":"#### Results","pinned":false,"hidden":false,"mode":"md"},{"id":432,"value":"Excitingly, all six benchmarks met our correctness criteria! Not surprising, and these six benchmarks only a portion of the program space that `cartokit`-generated programs can occupy, but hey, it's closing time on the semester.","pinned":false,"hidden":false,"mode":"md"},{"id":426,"value":"### Performance","pinned":false,"hidden":false,"mode":"md"},{"id":438,"value":"Let's move on to the more interesting evaluation—performance. On the performance front, I decided to conduct two studies:\n\n1. I benchmarked the run time performance of each backend's code generation algorithm on the six benchmark maps.\n2. I compared the bundle sizes and [largest contentful paint (LCP)](https://web.dev/articles/lcp) run times of the single-backend and multi-backend versions of `cartokit`.\n\nThe first experiment aimed to assess the overall performance of `cartokit`'s code generation while identifying whether any particular language-library backend degraded performance. The second experiment aimed to assess whether code splitting allowed us to preserve the bundle size and initial load performance of the single-backend architecture while opening up the functionality of multiple backends.","pinned":false,"hidden":false,"mode":"md"},{"id":613,"value":"I conducted all experiments in Google Chrome 130.0.6723.70 on a laptop running macOS 14.6.1 with a 2.3 GHz Quad-Core Intel Core i7 processor and 32GB RAM.","pinned":false,"hidden":false,"mode":"md"},{"id":565,"value":"#### Setup: Code Generation Performance","pinned":false,"hidden":false,"mode":"md"},{"id":568,"value":"To capture code generation run time's for each backend, I instrumented `cartokit`'s source code with calls to the browser's native [Performance API](https://developer.mozilla.org/en-US/docs/Web/API/Performance). Specifically, I used [`performance.mark`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/mark) to delineate start and end points for the algorithm's execution. For the starting point, I chose the program location _just before_ analysis of the `cartokit` DSL program began. For the ending point, I chose the program location _just after_ the final program string is generated and _just before_ it is returned to the caller of the algorithm.","pinned":false,"hidden":false,"mode":"md"},{"id":572,"value":"I recorded 10 executions of code generation for each of the four backends across each benchmark. This resulted in a total of 240 discrete code generation executions (40 executions per benchmark ${tex`\\times`} six benchmarks).","pinned":false,"hidden":false,"mode":"md"},{"id":506,"value":"#### Results: Code Generation Performance","pinned":false,"hidden":false,"mode":"md"},{"id":579,"value":"**Code generation was extremely consistent across backends**, with the following median run times for each backend across all benchmarks:\n\n1. MapLibre GL JS with JavaScript: **${d3.median(codegen.filter(d => d.backend === \"maplibre-javascript\"), d => d.duration).toFixed(2)}ms**\n2. Mapbox GL JS with JavaScript: **${d3.median(codegen.filter(d => d.backend === \"mapbox-javascript\"), d => d.duration).toFixed(2)}ms**\n3. MapLibre GL JS with TypeScript: **${d3.median(codegen.filter(d => d.backend === \"maplibre-typescript\"), d => d.duration).toFixed(2)}ms**\n4. Mapbox GL JS with TypeScript: **${d3.median(codegen.filter(d => d.backend === \"mapbox-typescript\"), d => d.duration).toFixed(2)}ms**","pinned":false,"hidden":false,"mode":"md"},{"id":587,"value":"In many ways, this isn't surprising. The discrepancies between the implementations of our four backends are quite minor, and all backends currently use the same analysis procedure. Moreover, the benchmarks do not exercise some of the more involved TypeScript code generation (e.g., typing user-defined functions, functions for asynchronous data fetching), meaning that, in many cases, they are identical to the generated JavaScript programs (thanks to type inference). Figure 5 below provides strip plots showing the distribution of code generation run times across each backend and each benchmark. Very consistent, much wow.","pinned":false,"hidden":false,"mode":"md"},{"id":517,"value":"$39","pinned":false,"hidden":false,"mode":"ojs"},{"id":597,"value":"

Figure 5. Strip plots showing code generation algorithm performance across benchmark and backend. Each subplot shows all 10 executions of each backend on the specified benchmark. The x-axis shows code generation execution run time, in milliseconds.","pinned":false,"hidden":false,"mode":"md"},{"id":608,"value":"#### Setup: Bundle Size","pinned":false,"hidden":false,"mode":"md"},{"id":611,"value":"To measure bundle size for the multi-backend and single-backend versions of `cartokit`, I summed the sizes of all JavaScript modules requested by the client on initial load. This information was available in the **Network Tab** of the Chrome Developer Tools.","pinned":false,"hidden":false,"mode":"md"},{"id":450,"value":"#### Results: Bundle Size","pinned":false,"hidden":false,"mode":"md"},{"id":454,"value":"$3a","pinned":false,"hidden":false,"mode":"md"},{"id":617,"value":"#### Setup: Largest Contentful Paint","pinned":false,"hidden":false,"mode":"md"},{"id":620,"value":"Beyond bundle size, I also wanted to capture some explicit metric of page load speed across the single-backend and multi-backend approaches. I decided to use Largest Contentful Paint (LCP) for this purpose, which captures the render time of the largest block-level element with text children (relative to when the user navigates to the page). In our case, the largest element fulfilling this requirement is the map itself (or, more precisely, the `div` element with `id=\"map\"`).\n\nTo collect LCP values, I ran [Lighthouse](https://developer.chrome.com/docs/lighthouse/overview) 10 times each on the production builds of both the single-backend and multi-backend versions of `cartokit`.","pinned":false,"hidden":false,"mode":"md"},{"id":472,"value":"#### Results: Largest Contentful Paint","pinned":false,"hidden":false,"mode":"md"},{"id":481,"value":"$3b","pinned":false,"hidden":false,"mode":"md"},{"id":490,"value":"lcpStripPlot = Plot.plot({\n height: 240,\n x: {\n axis: \"top\",\n grid: true,\n domain: [0, 1],\n label: \"LCP (s)\"\n },\n fy: {\n grid: true,\n tickFormat: (d) => (d === \"multiple\" ? \"Multi-Backend\" : \"Single-Backend\"),\n label: null,\n padding: 0\n },\n facet: {\n data: lcp,\n y: \"condition\",\n marginLeft: 120\n },\n marks: [\n Plot.dot(\n lcp,\n Plot.dodgeY(\n {\n anchor: \"middle\"\n },\n {\n x: \"lcp\",\n fill: \"condition\",\n fillOpacity: 0.25,\n stroke: \"condition\",\n r: 3\n }\n )\n )\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":736,"value":"

Figure 6. Strip plots showing LCP values across the multi-backend and single-backend architectures. Each dot represents one Lighthouse report value. The x-axis shows LCP time, in seconds.","pinned":false,"hidden":false,"mode":"md"},{"id":739,"value":"## Conclusion","pinned":false,"hidden":false,"mode":"md"},{"id":742,"value":"$3c","pinned":false,"hidden":false,"mode":"md"},{"id":38,"value":"## Appendix","pinned":false,"hidden":false,"mode":"md"},{"id":40,"value":"cartokitDemo = FileAttachment(\"cartokit-channels-min.mp4\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":361,"value":"codeSplittingDemo = FileAttachment(\"code-splitting-min.mp4\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":60,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":508,"value":"codegen = FileAttachment(\"codegen.json\").json()","pinned":true,"hidden":false,"mode":"ojs"},{"id":255,"value":"lcp = FileAttachment(\"cartokit-backend-metrics.json\").json()","pinned":true,"hidden":false,"mode":"ojs"},{"id":317,"value":"import { Editor } from \"@cmudig/editor\"","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W3d","stdlib":"1","titling":"auto"}