30:Tee0,# TON Cell parser done right [GitHub repo](https://github.com/qbane/ton-cell-parser). [npm](https://www.npmjs.com/package/@qbane/ton-cell-parser). Parse a TON cell by a format string. The usage is simple: Paste the content of a cell. Look up the matching specifier for each part, and type them in the format string field in the order they appear in the cell to unpack the data. Inspired by Python's [struct](https://docs.python.org/3/library/struct.html) module. A tutorial is given after the form. ## Syntax Comment: Java-style; Use `//` for line comments, `/*` .. `*/` for block comments. Integral types: * "xx`b`" = xx bits, defaults to `1`; returns a `BitString`’s string repr. * "xx`c`" = xx bytes, defaults to `1`; returns a hex string (technical: does not use `loadBuffer` under the hood because [`Uint8Array#toHex`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array/toHex) is not popular enough) * "xx`i`"/"xx`u`" = xx bits as signed/unsigned int, defaults to `32`; returns a number or BigInt when ≥ `54` bits **XXX**: maybe add type-accurate variants * "xx`vi`"/"xx`vu`" = xx-bit-length-prefixed signed/unsigned int, xx is required; returns a number or BigInt when ≥ `3` bytes * `h`/`H` = 16-bit signed/unsigned integer (short; shortcut to `16i`/`16u`) * `q`/`Q` = 64-bit signed/unsigned BigInt (qword; shortcut to `64i`/`64u`) Other data types: * `C` = coins (wire format `4vu`) * `A` = address (wire format `267b` but with format check -- use `?A` when it is nullable) * `B` = a single bit, but return the result as boolean * `(` ... `)` = parse into a sub-array; useful for grouping or acting as a placeholder * `D{`kkk [ `,`vvv ] `}` = dict with its key and value (optional) types respectively * `D*{`kkk [ `,`vvv ] `}` = dict direct; [see the implementation](https://github.com/ton-core/ton-core/blob/e0ed819973daf0484dfbacd0c30a0dcfe4714f8d/src/dict/Dictionary.ts#L260) * `E{`xxx`,`yyy`}` = either xxx (0-side) or yyy (1-side) * `E^{`xxx`}` = either xxx or (ref of xxx) Trailing: * `s` = parse the remaining bytes as a UTF-8 string tail; should be invoked at the end of a slice * “...`_`” = dump all the remaining bits as a bit string for debugging purposes * “...`$`” = assert the end of a slice (i.e., `endParse`, errs if there is remaining bits/refs to parse) Modifiers: (Must follow this order, and immediately follows a type specifier) * “`*`xxx” = read and discard (use `*`xx`b` as a way to skip xx bits) * “`?`xxx” = maybe xxx * “`^`xxx” = ref of xxx * **TODO** “`~(`xxx`)`” = spread the result of slice xxx (e.g., `X ~(YZ) W` -> `[x, y, z, w]`) ## More TODOs * Position info for error reporting * Format string pretty-printing or zipping * Code generation * xxd-like visualizations * Serializable vs. richer format (e.g., `BitString`, `Buffer`, `Address` instances) ## Experimental static type inference Docs are not available at this moment but you can try it out in the [✨ **TypeScript Playground** ✨](https://www.typescriptlang.org/play/?target=9&jsx=1&module=3#code/JYWwDg9gTgLgBAbzgSQHYDMCmUA0cCqqYAhgMYDWASpgM4CuANvAL5zpQQhwDkAAgI4AjYqkwB6GBFQBaUpgYNpJKDWzcAUOtJSa8dNBDEYAZRhRgqAOZwAvHAAGpAEIBhAHoAKAIIuAlOoAmABY6JydAkLgo6JjYuKixMTgAbQA3AOkpBgBPAF04C10ReAB3YBgACwATKGIS4gY2TEx1AGYA0niursS4VIbgOFV4CuIaCvVPdSjPABEEAIBWADZBHBcXZjo6DZdXXxje0jooKDgOOlQquCqjYmm4OYWVtY2tnY39w6SwKExU84QS7XW4we7+DbBOjqFykZadbqIhJJNIZLJ5SYeJHYuC9DgMWgPLxBAAcdGJZK8D08Xi8cDpYgAVCl0plUDl8vQqhBsHBGWJ-P5PJ5fCKRQccUcIFVCfZNL0ygo4AA5ADyABVzpd1FLULobrRSDRbHBlKoAGIGIymcxWDz6KCGExmCyWfy6-U0BjAOQm3S2yzqiAueQMDz2ey+AB0gkwlgsAAViCpMB5-No9fAaMa7AhWGMCEQyFRaIwYAAeNBYKDlmDZMCYCDoNhW50BgB87a0Ong5UwIBcQNQMAAXHAAIwkv00KOXEgUTBVKMEqyVbuZuDHU6UIdVWZ3MdIOsNsfcKo+mDcOCsOzZ2dFhdVZIATlyyQADG-P+ogA). 31:T990,## Tutorial Note: It is recommended that you copy this text elsewhere for reference. Let's get started by parsing a real-world Jetton transfer message. Take this Jettons transfer transaction for example. You can see the **internal message** by clicking **Show details**, and then clicking **Copy Raw body** [sic], this copies the body cell as a hex string `b5ee9c72`... We are going to come up with a format string for decoding this message. According to the [message layout](https://docs.ton.org/v3/guidelines/dapps/asset-processing/jettons#message-layouts), or the [TL-B schema](https://github.com/ton-blockchain/TEPs/blob/master/text/0074-jettons-standard.md#internal-message-handlers), you can unpack it with the format string as follows: ```java 4c // op code, 4 bytes; here we expect "0F8A7EA5" Q // query_id, aka uint64 or "64u" C // amount, coins A // destination, address ?A // response_destination, a "maybe-address" (empty in this case) ?^() // custom_payload, maybe-ref (also empty in this case) C // forward_ton_amount, coins ``` You can omit all whitespaces and comments, and just type `4cQCA?A?^()C`. This makes your format string more like a cryptic regular expression (so bad). ### Going further You will notice that the data has not ended yet (if you add `$` at the end you will receive an error). The reason is that this message was meant for swapping on [STON.fi](https://ston.fi) and there was a `forward_payload` left to be parsed. Refer to the message layout it is of an `Either Cell ^Cell` type, but let us pretend that the parser does not know about an `Either` type. For a fixed message, we can dissect it by using the wire format nonetheless. *Append* the following: ```java B // pause here: observe what is parsed // ^() // parse the referenced slice (placeholder) ``` The `B` takes a bit. In this case, it's `true` (or a `1` bit), which corresponds to the "right" side of the Either, indicating a `^Cell` follows. Type `^()` to extract the content. You can refer to STON.fi's [documentations](https://docs.ston.fi/document/developer-section/api-reference-v1/router#swap-0x25938561) to figure out a format string such like `4c A C A ?(A)` to be put between the parentheses. This completes the parsing. 32:[{"id":"a685364e44b3798964575049adb2de70b3640f123c8d46e1513ff6d5081147d99a926072ca01156260fb871fc94a7e2698e0d4a6f85fe5541c6ae6b1cf42847b","href":"https://static.observableusercontent.com/files/a685364e44b3798964575049adb2de70b3640f123c8d46e1513ff6d5081147d99a926072ca01156260fb871fc94a7e2698e0d4a6f85fe5541c6ae6b1cf42847b","name":"sample-data@1.txt","type":"text/plain","lastModified":1745606581653.525,"size":7748}] 2e:{"title":"TON Cell parser done right","cells":[{"id":0,"value":"$30","pinned":false,"hidden":false,"mode":"md"},{"id":18,"value":"demoData = FileAttachment(\"sample-data@1.txt\").text()","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":"viewof cellStr = Inputs.text({ label: 'Cell in hex or base64', value: demoData })","pinned":false,"hidden":false,"mode":"ojs"},{"id":11,"value":"viewof formatString = Inputs.textarea({ label: 'Format string', value: `\\\ncBC^(AC)\n24uBB\n24u // [v2-only] instant withdrawal fee\n32c // vali set hash\n^(\n ^(D{256b,CC}uuCCCBC) // curr round data\n ^(D{256b,CC}uuCCCBC) // prev round data\n)\nCC24u\nCc6c // [v2-only]\n^( // roles\n A48uA48uA\n ^(AA A /* [v2-only] sudoer */)\n)\n^(^()^()^()) // codes\n`, rows: 8, monospace: true })","pinned":false,"hidden":false,"mode":"ojs"},{"id":102,"value":"{\n const [err, result] = decoded\n if (err) { return html`

### Unpack error ###
${err}

` }\n\n const { unpacked, remainingBits, remainingRefs } = result\n \n let timer\n const copyButton = Inputs.button('Copy as JSON', { value: null, reduce: () => {\n const data = JSON.stringify(unpacked, (z, v) => typeof v === 'bigint' ? v.toString() : v, 2)\n navigator.clipboard.writeText(data)\n copyButton.textContent = 'Copied!'\n if (timer) clearTimeout(timer)\n timer = setTimeout(() => {\n copyButton.textContent = origText\n timer = null\n }, 1000)\n }}).querySelector('button')\n const origText = copyButton.textContent\n \n return md`## Parsed cell content\n**Remaining bits**: \\`${remainingBits}\\` **Remaining refs**: \\`${remainingRefs}\\` ${copyButton}\n

${expandInspect(inlineInspect(unpacked), 2)}

\n`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":72,"value":"{\n const formatStringSection = expandInspect(inlineInspect(decoded[0] ? html`

${decoded[0].message}

` : decoded[1]), 1)\n const emittedTokensSection = Inputs.table(mutable emittedTokens.map(({pos, token: {type, id, ...token}}) => ({pos, type, id, token})), {\n rows: Infinity,\n format: { token: inlineInspect },\n width: { pos: '2em', type: '6em', id: '6em' },\n })\n return md`## Parsed format string\n

AST:${formatStringSection}

Tokens:${emittedTokensSection}

`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":13,"value":"decoded = {\n try {\n const result = parseCell(\n stringToCell(cellStr).beginParse(),\n parseFormatString(formatString),\n { DictionaryKeyClassMap: TON.Dictionary.Keys })\n return [null, result]\n } catch (e) { console.error(e); return [e, undefined] }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":488,"value":"mutable emittedTokens = []","pinned":false,"hidden":false,"mode":"ojs"},{"id":624,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":68,"value":"function parseFormatString(str) {\n mutable emittedTokens = []\n const [err, result, tlist] = TonCellParser.getEmittedTokens(() => parseFormatStringImpl(str))\n // make it possible to view (incomplete) token list on parsing error\n mutable emittedTokens = tlist\n if (err) throw err\n return result\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":15,"value":"parseCell = TonCellParser.decodeSlice","pinned":true,"hidden":false,"mode":"ojs"},{"id":774,"value":"parseFormatStringImpl = TonCellParser.parseFormatString","pinned":true,"hidden":false,"mode":"ojs"},{"id":41,"value":"function stringToCell(str) {\n return TonCellParser.stringToCell(str, TON.Cell)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":190,"value":"---","pinned":false,"hidden":false,"mode":"md"},{"id":192,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":182,"value":"---","pinned":false,"hidden":false,"mode":"md"},{"id":741,"value":"TonCellParser = import('https://esm.run/@qbane/ton-cell-parser@v0.0.1-beta.12')","pinned":true,"hidden":false,"mode":"ojs"},{"id":4,"value":"import { TON } from '@qbane/ton'","pinned":true,"hidden":false,"mode":"ojs"},{"id":218,"value":"import { inlineInspect, expandInspect } from '@observablehq/fabians-toolbox'","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W32","stdlib":"1","titling":"auto"}