32:Tb7e,{ const code = s => `${s.replace(/ /g,' ')}`; const el = md` Clojure is designed to “embrace the host platform”. For example, a function in ClojureScript is also a function in JavaScript and is interchangeable in both contexts. Recently, JavaScript has added newer function types that are _“parkable”_: - ES2015 [Generators] - parked with __yield__ - ES2017 [Async functions] - parked with __await__ - ES2018 [Async generators] - parked with __yield__ + __await__ ClojureScript does not allow access to these native control flows—not even with [macros](#macros). Thus, this document explores how new function _metadata_ and _special forms_ can compile to these native semantics, which GClosure can transpile when needed: JavaScript feature | ClojureScript proposal | JavaScript ---------------------|---------------------------------------------|---------------------------------------- [Generators] | ${code(`(defn ^:js-gen foo [] ...)`)} | ${code(` function* foo() {...}`)} [Async Functions] | ${code(`(defn ^:js-async foo [] ...)`)} | ${code(`async function foo() {...}`)} [Async Generators] | ${code(`(defn ^:js-async-gen foo [] ...)`)} | ${code(`async function* foo() {...}`)} anon generator | ${code(` (fn ^:js-gen [] ...)`)} | ${code(` function* () {...}`)} anon async function | ${code(` (fn ^:js-async [] ...)`)} | ${code(`async function () {...}`)} anon async generator | ${code(` (fn ^:js-async-gen [] ...)`)} | ${code(`async function* () {...}`)} [Yield (bidirectional)][yield] | ${code(`(js-yield ...)`)} | ${code(`yield ...;`)} [Yield (iterate)][yield*] | ${code(`(js-yield* ...)`)} | ${code(`yield* ...;`)} [Await][await] | ${code(`(js-await ...)`)} | ${code(`await ...;`)} [Generators]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function* [Async functions]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function [Async generators]:https://github.com/tc39/proposal-async-iteration#async-generator-functions [yield]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/yield [yield*]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/yield* [await]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await [for await]:https://github.com/tc39/proposal-async-iteration#the-async-iteration-statement-for-await-of [core.async]:https://www.braveclojure.com/core-async/ [lazy-seq]:http://www.thesoftwaresimpleton.com/blog/2014/09/08/lazy-seq/ `; el.querySelector('table').className = "proposal"; return el; }33:T5f7,usecases = md`## Use Cases _who wants what and why?_ __yield__ - [Some users](https://dev.clojure.org/jira/browse/CLJS-1986) have asked for this, without specific use-cases. __await__ - [Lumo users](https://github.com/anmonteiro/lumo/issues/233) have asked for this, presumably to adhere to Node.js idioms. __await+yield__ - I've asked for this 🙂—specifically to _script async events with \`await\`s and to allow early cancelation at \`yield\`s_. This method is popular among game developers, and was invented in the Unity game engine as [Coroutines] in 2006 ^{[[1][1]][[2][2]]}. This method was later rediscovered as the driving construct of [Observable Notebooks]. Both were created by plugging _generators_ into custom async runtimes, but they are now adopted as language-level features by [Python], [C#], [Hack], [JavaScript]. [Coroutines]:https://docs.unity3d.com/Manual/Coroutines.html [Python]:https://www.python.org/dev/peps/pep-0525/#asynchronous-generators [C#]:https://github.com/dotnet/csharplang/blob/master/proposals/async-streams.md#syntax-1 [JavaScript]:https://github.com/tc39/proposal-async-iteration#async-generator-functions [Hack]:https://docs.hhvm.com/hack/async/generators [1]:https://web.archive.org/web/20061209063155/http://unity3d.com:80/Documentation/ScriptReference/index.Coroutines_26_Yield.htm [2]:https://www.gamedev.net/forums/topic/374644-thoughts-on-coroutines-for-game-scripting/ [Observable Notebooks]:https://beta.observablehq.com/@mbostock/introduction-to-generators `34:T452,md`## What kind of JS will be produced?

_If \`:optimizations :none\` is specified, the new syntax will be produced as expected, otherwise the following applies:_ ClojureScript compiles to JS directly, but this output is often optimized by the Google Closure Compiler. To make this optimization work, ~~the [\`language-in\`] option must be set appropriately for Closure to recognize the dialect~~ (as [pointed out](https://groups.google.com/d/msg/clojurescript/scUMhU-ctEM/fLRYocouDQAJ), this is auto-detected): - Generators: \`ecmascript-2015\` or higher - Async functions: \`ecmascript-2017\` or higher - Async generators: \`ecmascript-2018\` or higher Closure Compiler then __transpiles__ to any dialect you want using [\`language-out\`]. In the future, as [pointed out](https://groups.google.com/d/msg/clojurescript/scUMhU-ctEM/fLRYocouDQAJ), this will be replaced with another option letting you specify individual feature sets to transpile.) [\`language-in\`]:http://cljs.github.io/api/compiler-options/language-in [\`language-out\`]:http://cljs.github.io/api/compiler-options/language-out `35:T47f,macros = md`## Just use macros?

Macros cannot be employed to allow direct access to JavaScript's native control flows for yield/await—since none of ClojureScript's special forms use them. But they can be approximated by implementing similar constructs. These libraries provide some relevant macros (via this [blogpost](https://blog.jeaye.com/2017/09/30/clojurescript-promesa/)): - [promesa] - provides [\`async\`/\`await\` macros](http://funcool.github.io/promesa/latest/#async-await-syntax) - [cljs-promises] - provides \`(

...
(function(){   // <-- inserted by compiler
  yield ...;   // <-- error (must be inside function*)
  await ...;   // <-- error (must be inside async function)
}()

As [pointed out][1], this is not straightforward to implement currently. IIFEs are sometimes wrapped around inner expressions for safe scoping. So any \`yield\` or \`await\` caught inside one of these IIFEs would throw an error. If no value is returned from the IIFE, a block-scope would be sufficient:

{
  yield ...; // <-- fine if block is inside function*
  await ...; // <-- fine if block is inside async function
  // no value return
}

But in the more likely case that a value is returned from the IIFE, we may have to wait for [do-expressions] to land in JS, depending on whether it will have [Block-like or IIFE-like behavior](https://github.com/tc39/proposal-do-expressions/issues/5):

do {
  yield ...; // <-- fine if "do" has block behavior, and "do" is inside function*
  await ...; // <-- fine if "do" has block behavior, and "do" is inside async function
  123;       // <-- "do" expression evaluates to last value
}

This is a list of IIFEs that are affected (there may be more): - [\`case*\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L540) - [\`def*\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L698) - [\`emit-apply-to\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L729) - [\`throw*\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L564) - [\`emit-variadic-fn-method\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L802) - [\`fn*\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L857) - [\`do\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L927) - [\`try\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L938) - [\`emit-let\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L952) - [\`letfn\`](https://github.com/clojure/clojurescript/blob/r1.10.238/src/main/clojure/cljs/compiler.cljc#L992) [do-expressions]:https://github.com/tc39/proposal-do-expressions [CLJS-1986]:https://dev.clojure.org/jira/browse/CLJS-1986 [1]:https://groups.google.com/d/msg/clojurescript/scUMhU-ctEM/uH-YUIspDAAJ `37:T4b1,coreasync = md`## go-blocks vs async-generators

An interesting use-case—create an asynchronous function that can be canceled by its caller. Using a JavaScript async generator:

async function* foo() {
  for (let i=0; i<100; i++) {
    await delay(1000);
    yield i;
  }
}

const a = foo();
a.next();
a.next();
a.return(); // <-- shuts down the generator (at next yield?)

Using a Clojure core.async go-block (_thanks to [thheller](https://clojureverse.org/t/async-generator-functions-in-cljs-requesting-feedback/2262/2?u=shaunlebron)_):

(defn foo []
  (let [ch (async/chan)]
    (go (loop [i 100]
          (when (pos? i)
            (<! (async/timeout 1000))
            (when (>! ch i) ;; <-- stop if caller closed the channel
              (recur (dec i)))))
        ;; cleanup here
        (prn ::terminated))
    ch))

(let [ch (foo)]
  (go (prn (<! ch))
      (prn (<! ch))
      (async/close! ch))) ;; <-- signals to go-block to shut itself down

See [async+yield use cases](#usecases) for more references and history of this method. [Async iterators]:https://github.com/tc39/proposal-async-iteration `38:T1240,md`### Initial discussion with David

I proposed this feature to David on \`#cljs-dev\` on Clojurians Slack on Tuesday March 27, reproduced below. I want to summarize the discussion before starting the proposal: > __shaunlebron__: > I’ve been thinking about how to create JS generators and async functions in ClojureScript > (they are heavily used in https://observablehq.com notebooks, and I eventually want to see if it would be possible to use ClojureScript there) (edited) > I wonder if it would be a good idea to add \`:js-generator\` and \`:js-async\` metadata to functions so they can compile to \`function*\` and \`async function\`, respectively > and \`js-yield\`,\`js-yield*\`,\`js-await\` for emitting => \`yield\`,\`yield*\`,\`await\` > __dnolen__: > It’s been proposed before but I don’t see any benefit > Well not enough benefit to seriously consider their inclusion > __shaunlebron__: > js interop mainly, it’d be pretty ugly to use existing js generators and async functions inside cljs > i thought i could just reach for core.async instead, but I didn’t really understand generators/async functions until I saw them combined, which are now implemented in most browsers today: https://github.com/tc39/proposal-async-iteration#async-generator-functions (edited) > __dnolen__: > I don’t see why you would need them for interop > you don’t need core.async for interop > and they’re more or less like core.async - nobody uses that stuff unless they have to because callbacks are the baseline > they’re even less meaningful for React code (>70% of what ClojureScript users are doing) where async patterns are just callbacks > I’d like to see more compelling cases where they are solving some actual problem for end users before we start looking at adding more JS primitives > __rarous__: > @shaunlebron ES \`await\`/\`async\` is just sugar over functions that return \`Promise\`. You can always use the \`Promise\` API on the result of \`async function\`. If you need to provide \`async function\` in your cljs public API, just return \`Promise\`. Generators are sugar for generating stateful \`Iterator\`s and way to add some kind of laziness to JS. Cljs structures are lazy and implement \`Iterator\` for long time. Cljs is ahead in this. (edited) > __shaunlebron__: > thanks for replying. it would just be nice to drop down to JS things since clojure is about embracing the host (if I’m interpreting that goal correctly) > also, generators are pausable/cancelable coroutines that can send _and_ receive values at yield points. not sure a lazy sequence can do that > as for just using the Promise API, the synchronous style of async/await is nicer, similar to why core.async was created > @dnolen my feeling is that it’s more important than we realize, so I’ll do the work to make a case for it. already drafting a notebook that is essentially describing how it makes cancelable “goblocks” possible > __dnolen__: > @shaunlebron sure, but I think making a case for has to be grounded in the reality of the existing user base > this request comes up very infrequently and never with a actual use case that relates at all to how people are actually using ClojureScript (edited) > basically I’ve already rejected this at least 2 times over after past 3 years > and I haven’t seen anything happen in the JavaScript world that makes me think it’s delivering enough value to take it on > also the fact that we can process ES6 in build means you can do what you need in JS if it’s really a problem > just like Clojure there’s no reason to import all the primitives of the host, it’s not meaningful > sometimes you have to write some Java - so what? > __shaunlebron__: > yeah, that’s compelling > __dnolen__: > ES6 is really not that hard to write when you need it > goog require works > this was big part of the original idea of ES6 module processing in ClojureScript > write ES6 when you need to and it not suck > __shaunlebron__: > yeah, and that ES6 code can call cljs functions fine too > __dnolen__: > yes > __shaunlebron__: > this may be the direction that I end up agreeing with > __dnolen__: > but also it’s gives an escape hatch > they’re going to keep adding sugar / keywords to JavaScript (edited) > I don’t want us to be chasing that all the time > __shaunlebron__: > would age and increased use of generators and async/await functions merit their inclusion eventually? > __dnolen__: > @shaunlebron of course if people start clamoring for this feature because it’s a common requirement I don’t see why not > but I’m not seeing that today `39:T41a,md` > __bh__: > Lots of times importing core.async just for simple callback mgmt feels like way overkill imo > __orestis__: > I’d love async/await and generators in CLJS. Happy to read a draft on the proposal if it helps. I also received a DM on Clojurians Slack from Justin Lee: > __justinlee__: > Hey I saw a bit of your back-and-forth with dnolan regarding generators and async functions. I’m super glad you are taking on the case for them. > A couple of points to consider: first, code size. I don’t know for sure, but it would surely be smaller to use native generators + yield than have the entire core.async library + the generated state machine. The polyfill for generators looks just like the state machine go blocks generate. > Second point: error handling. Chrome does maintains callstacks across async boundaries. But core.async uses nextTick, which shears off the call stack. That means if the producer has an uncaught exception, it is very difficult to figure out. > Anyway, just thought I’d put in my 2 cents. `3a:[] 30:{"title":"Proposal - async/generator functions in ClojureScript","cells":[{"id":1172,"value":"md`# Proposal - async/generator functions in ClojureScript\n💬Join the discussion on [ClojureVerse](https://clojureverse.org/t/request-for-feedback-async-generator-functions-in-cljs/2262)\n\n[mailing list discussion]:https://groups.google.com/forum/#!topic/clojurescript/scUMhU-ctEM\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1153,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":928,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":31,"value":"md`\n\n## Code Examples\n

\nGenerators (with yield):\n\n

\n(defn ^:js-gen foo []\n  (doseq [i 100]\n    (println (js-yield i))) ;; <-- yield a value, then receive a value\n  (js-yield* #js[1 2 3]))\n

\n\n

\nfunction* foo() {\n  for (let i=0; i<100; i++) {\n    console.log(yield i);   // <-- yield a value, then receive a value\n  }\n  yield* [1, 2, 3];\n}\n

\n\nAsync functions (with await):\n\n

\n(defn ^:js-async foo []\n  (doseq [i promises]\n    (js-await i)))\n

\n\n

\nasync function foo() {\n  for (let i of promises) {\n    await i;\n  }\n}\n

\n\nAsync generators (with yield _and_ await):\n\n

\n(defn ^:js-async-gen foo []\n  (doseq [i promises]\n    (js-yield (js-await i))))\n

\n\n

\nasync function* foo() {\n  for (let i of promises) {\n    yield await i;\n  }\n}\n

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":508,"value":"md`## The \\`js-\\` naming convention\n\nNames are chosen based on the following precedents:\n\n- [js-arguments](http://cljs.github.io/api/cljs.core/js-arguments)\n- [js-comment](http://cljs.github.io/api/cljs.core/js-comment)\n- [js-debugger](http://cljs.github.io/api/cljs.core/js-debugger)\n- [js-delete](http://cljs.github.io/api/cljs.core/js-delete)\n- [js-in](http://cljs.github.io/api/cljs.core/js-in)\n- [js-inline-comment](http://cljs.github.io/api/cljs.core/js-inline-comment)\n- [js-invoke](http://cljs.github.io/api/cljs.core/js-invoke)\n- [js-keys](http://cljs.github.io/api/cljs.core/js-keys)\n- [js-mod](http://cljs.github.io/api/cljs.core/js-mod)\n- [js-obj](http://cljs.github.io/api/cljs.core/js-obj)\n- [js-reserved](http://cljs.github.io/api/cljs.core/js-reserved)\n- [js-str](http://cljs.github.io/api/cljs.core/js-str)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":148,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":357,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":101,"value":"md`## Why should we include it?\n\n- __Interop__: When ClojureScript interops with JS libraries, it should be able to produce and consume generators and await functions in a way that isn't awkward. It is possible to do this without the syntax, but it can get unwieldy—for the same reasons JS added the more ergonomic syntax. This would be particularly useful in [Node.js](https://github.com/anmonteiro/lumo/issues/233)\n- __Fundamental primitives__: Async functions have become a fundamental primitive in dealing with promises and callback hell in JS, and generators are fundamental to [new asynchronous methods]. It might make sense for ClojureScript to take advantage of the paradigms of its host platform.\n\n[new asynchronous methods]:https://beta.observablehq.com/@mbostock/introduction-to-generators\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":103,"value":"md`## Why should we NOT include it?\n- __Avoid accretion__: JavaScript syntax will continue to grow, and its effect on ClojureScript syntax accretion must be managed.\n- __Escape hatch already available__: ClojureScript already allows you to include ES6 files in your project, so this should be your escape hatch if you want to use extra JS syntax.\n- __Not commonly used yet?__: might make sense to wait for this until a critical mass of users are needing to use it in their projects.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":442,"value":"blockers = md``","pinned":false,"hidden":false,"mode":"ojs"},{"id":204,"value":"$36","pinned":false,"hidden":false,"mode":"ojs"},{"id":165,"value":"md`## Why not core.async?\n

\nAddressing the common argument that \\`core.async\\` obviates the need for JS async/await and yield:\n\n- __Ergonomics__: It seems unwieldy to wrap promise-yielding functions in go-blocks or channel-yielding function wrappers, when JS has a better facility for dealing with them.\n- ~~__Debugging__: It's easier to debug a native async function instead of a compiled core.async go-block. Call stacks are preserved when using async functions, but not with go-blocks~~\n - _well actually, this is [not true anymore in latest chrome!](https://bugs.chromium.org/p/chromium/issues/detail?id=661705)_\n- __It's just different__: their models are similar, but they have different goals.\n`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":292,"value":"$37","pinned":false,"hidden":false,"mode":"ojs"},{"id":154,"value":"md`## Appendix`","pinned":false,"hidden":false,"mode":"ojs"},{"id":853,"value":"md`### Related discussions\n\n- https://clojureverse.org/t/any-syntax-to-match-async-await-in-creating-web-servers/825\n- https://news.ycombinator.com/item?id=10800128\n- https://groups.google.com/forum/#!topic/clojurescript/LBy0yiZiWrA\n- Request in Lumo: https://github.com/anmonteiro/lumo/issues/233\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":"$38","pinned":false,"hidden":false,"mode":"ojs"},{"id":21,"value":"md`### Community feedback\n\nI wanted to get some preliminary thoughts and gauge interest on twitter:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":23,"value":"html`\n

Drafting a proposal for why it should probably be possible to create Generators and Async/Await functions in ClojureScript. (Have you found yourself wanting to do this?) Their latest versions essentially allow cancelable "go-blocks", which is something I've wanted for years.
— Shaun Lebron (@shaunlebron) March 27, 2018

\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":26,"value":"$39","pinned":false,"hidden":false,"mode":"ojs"},{"id":729,"value":"md`## Proposal Updates\n\n- 2018-06-11 - add use-case links\n- 2018-05-31 - \\`:language-out\\` will be replaced soon\n- 2018-05-31 - \\`:language-in\\` is auto-detected\n- 2018-05-31 - (chrome preserves core.async call stacks now)\n- 2018-05-31 - add \\`:js-async-gen\\` (async generators)\n- 2018-05-31 - shorten \\`:js-generator\\` \\`->\\` \\`:js-gen\\`\n- 2018-05-30 - add anon forms to table\n- 2018-05-25 - add section on macros\n- 2018-05-24 - add section on iife problems and core.async\n- 2018-05-24 - first draft\n- 2018-03-27 - discussion with david\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":467,"value":"codeStyles = html`\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":460,"value":"md`*Thanks for the Continuous Worm Conveyor, [ClipArt ETC](http://etc.usf.edu/clipart/59000/59007/59007_cont-convey.htm).*`","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W3a","stdlib":"1","titling":"auto"}