{"action":"created","issue":{"url":"https://api.github.com/repos/WordPress/gutenberg/issues/8244","repository_url":"https://api.github.com/repos/WordPress/gutenberg","labels_url":"https://api.github.com/repos/WordPress/gutenberg/issues/8244/labels{/name}","comments_url":"https://api.github.com/repos/WordPress/gutenberg/issues/8244/comments","events_url":"https://api.github.com/repos/WordPress/gutenberg/issues/8244/events","html_url":"https://github.com/WordPress/gutenberg/issues/8244","id":345099403,"node_id":"MDU6SXNzdWUzNDUwOTk0MDM=","number":8244,"title":"Overview of Short-term Parsing Enhancements","user":{"login":"mcsf","id":150562,"node_id":"MDQ6VXNlcjE1MDU2Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/150562?v=4","gravatar_id":"","url":"https://api.github.com/users/mcsf","html_url":"https://github.com/mcsf","followers_url":"https://api.github.com/users/mcsf/followers","following_url":"https://api.github.com/users/mcsf/following{/other_user}","gists_url":"https://api.github.com/users/mcsf/gists{/gist_id}","starred_url":"https://api.github.com/users/mcsf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcsf/subscriptions","organizations_url":"https://api.github.com/users/mcsf/orgs","repos_url":"https://api.github.com/users/mcsf/repos","events_url":"https://api.github.com/users/mcsf/events{/privacy}","received_events_url":"https://api.github.com/users/mcsf/received_events","type":"User","site_admin":false},"labels":[{"id":562969841,"node_id":"MDU6TGFiZWw1NjI5Njk4NDE=","url":"https://api.github.com/repos/WordPress/gutenberg/labels/Framework","name":"Framework","color":"fbca04","default":false},{"id":596962405,"node_id":"MDU6TGFiZWw1OTY5NjI0MDU=","url":"https://api.github.com/repos/WordPress/gutenberg/labels/[Component]%20Parsing","name":"[Component] Parsing","color":"fbca04","default":false},{"id":531680824,"node_id":"MDU6TGFiZWw1MzE2ODA4MjQ=","url":"https://api.github.com/repos/WordPress/gutenberg/labels/[Type]%20Task","name":"[Type] Task","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":{"login":"mcsf","id":150562,"node_id":"MDQ6VXNlcjE1MDU2Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/150562?v=4","gravatar_id":"","url":"https://api.github.com/users/mcsf","html_url":"https://github.com/mcsf","followers_url":"https://api.github.com/users/mcsf/followers","following_url":"https://api.github.com/users/mcsf/following{/other_user}","gists_url":"https://api.github.com/users/mcsf/gists{/gist_id}","starred_url":"https://api.github.com/users/mcsf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcsf/subscriptions","organizations_url":"https://api.github.com/users/mcsf/orgs","repos_url":"https://api.github.com/users/mcsf/repos","events_url":"https://api.github.com/users/mcsf/events{/privacy}","received_events_url":"https://api.github.com/users/mcsf/received_events","type":"User","site_admin":false},"assignees":[{"login":"mcsf","id":150562,"node_id":"MDQ6VXNlcjE1MDU2Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/150562?v=4","gravatar_id":"","url":"https://api.github.com/users/mcsf","html_url":"https://github.com/mcsf","followers_url":"https://api.github.com/users/mcsf/followers","following_url":"https://api.github.com/users/mcsf/following{/other_user}","gists_url":"https://api.github.com/users/mcsf/gists{/gist_id}","starred_url":"https://api.github.com/users/mcsf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcsf/subscriptions","organizations_url":"https://api.github.com/users/mcsf/orgs","repos_url":"https://api.github.com/users/mcsf/repos","events_url":"https://api.github.com/users/mcsf/events{/privacy}","received_events_url":"https://api.github.com/users/mcsf/received_events","type":"User","site_admin":false},{"login":"mtias","id":548849,"node_id":"MDQ6VXNlcjU0ODg0OQ==","avatar_url":"https://avatars2.githubusercontent.com/u/548849?v=4","gravatar_id":"","url":"https://api.github.com/users/mtias","html_url":"https://github.com/mtias","followers_url":"https://api.github.com/users/mtias/followers","following_url":"https://api.github.com/users/mtias/following{/other_user}","gists_url":"https://api.github.com/users/mtias/gists{/gist_id}","starred_url":"https://api.github.com/users/mtias/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtias/subscriptions","organizations_url":"https://api.github.com/users/mtias/orgs","repos_url":"https://api.github.com/users/mtias/repos","events_url":"https://api.github.com/users/mtias/events{/privacy}","received_events_url":"https://api.github.com/users/mtias/received_events","type":"User","site_admin":false}],"milestone":null,"comments":5,"created_at":"2018-07-27T05:53:33Z","updated_at":"2018-09-04T10:59:55Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"One of the foundational pieces of this project, the block-based language of Gutenberg has so far been implemented in the system by a set of parsers generated from a [PEG grammar](https://github.com/WordPress/gutenberg/blob/b4f33279f57e668835d677079691b2c16c98c500/packages/block-serialization-spec-parser/grammar.pegjs) describing the language.\r\n\r\nTwo motivating factors to back this emerged early on in the development cycle: i) a PEG grammar is easy to understand, review and maintain; ii) the parsers [generated](https://github.com/WordPress/gutenberg/blob/b4f33279f57e668835d677079691b2c16c98c500/bin/create-php-parser.js) from it, one in JS and one in PHP, worked out of the box, and — with caveats — this proved adequate even as the project grew: for instance, the parsing stack easily accommodated block nesting.\r\n\r\nThis intelligibility and [quasi-self-documenting](https://github.com/WordPress/gutenberg/pull/6116) character of the PEG grammar are what led to establishing it as the official specification of the language of Gutenberg in the form of [its own package](https://github.com/WordPress/gutenberg/pull/7664). This is an important step for the long-term success of Gutenberg as not just an editor, but a multi-environment ecosystem bound by the same language.\r\n\r\n## Opportunities\r\n\r\nAs Gutenberg braces for merging into core, all the tooling surrounding blocks needs to be as robust and versatile as any other part of WordPress. Among other things, this means that webmasters, plugin authors and platform engineers should have the power to hot-swap parsers to optimize for given requirements — whether requirements concern block analysis, hardware constraints, runtime, etc. Past experiments have confirmed that the PEG grammar is [easily tweaked](https://github.com/WordPress/gutenberg/pull/6170) to output specific block information and can be optimized if told to ignore other aspects.\r\n\r\nFurthermore, we have a duty to perform as well as possible. Our auto-generated parsers are known not to perform as well as other systems, a trade-off we made for clarity. Client-side parser performance has so far been satisfying, but the server-side parser is a known bottleneck and may not be suitable for demanding posts. The server-side issue was mitigated by [altogether eschewing](https://github.com/WordPress/gutenberg/pull/4591) the PHP parser when rendering front-end pages in favor of a simpler parse, but the fact is that obtaining a full parse tree of a post on the server should be performant and robust.\r\n\r\nFinally, Gutenberg blocks are to reach well beyond WP-Admin and base WordPress installations. Namely, any third-party clients should be able to work with Gutenberg blocks, from the mobile apps to any community tool. This requirement implies that the language of Gutenberg will need to be parsed in diverse environments and runtimes, well beyond the scope of the base Gutenberg project.\r\n\r\nThus, there is the opportunity to establish the language of Gutenberg as a _de facto_ standard for semantic content formatting. Perhaps choosing parsers will be similar to choosing database systems: most site owners will go with the default, such as MySQL, but competing software abounds for those with different needs.\r\n\r\n## Parser tooling\r\n\r\nCloses #6994.\r\n\r\nBefore we look at parsing itself, we need to be able to develop, validate and compare implementations.\r\n\r\n**Develop.** Any party should be able to understand the specified language. For this, it should be clear that the PEG grammar is intended to be used as a reference. Tasks:\r\n- [x] Offer PEG grammar and its parser as its own `spec-parser` package. #7664 \r\n- [x] Auto-generate human-readable version of block grammar. #6116\r\n- [ ] Increase visibility of this reference, perhaps via a standalone documentation page.\r\n\r\n**Validate.** Developers should be provided official tools that can, as best as possible, test for correctness of parsers. Tasks:\r\n- [ ] Develop a large and diverse corpus, comprising posts long and short, with many and few blocks, different kinds of content, written in different locales, and with documents full of various kinds of invalid blocks.\r\n  - [x] @dmsnell has started this in his repository [gutenberg-document-library](https://github.com/dmsnell/gutenberg-document-library).\r\n- [ ] Provide easy-to-use validators that test against the corpus.\r\n  - [ ] Add a JEST test to build and compare language-agnostic parsers. #6030\r\n\r\n**Compare.** A parser doesn't just need to work, it likely needs to perform well. To help inform development (and instigate a healthy sense of competition?), it should be possible to compare parser performance side-by-side or multilaterally. Here, performance encompasses both space and time. Tasks:\r\n- [ ] Parser: Build system to compare alternative parser implementations. #6831\r\n\r\nUltimately, all tasks for the above paragraphs should converge towards providing:\r\n- [ ] A standalone reference Web page providing spec documentation and validation-comparison tools. Tentatively: https://wordpress.github.io/gutenberg/\r\n\r\nRelated open issues:\r\n- [ ] Check support for `/u` flag in installed PCRE library #4852\r\n- [ ] Verify parser doesn't cause PCRE fault with Subresource Integrity (SRI) Manager plugin and PHP 5.6.36 #8671\r\n\r\n## Exploring new parsers\r\n\r\nThe past months have seen fascinating and encouraging developments in the development of competing parsers.\r\n\r\n- @Hywan developed a [Rust parser](\r\nhttps://github.com/Hywan/gutenberg-parser-rs/) that compiles to a number of different [targets](\r\nhttps://github.com/Hywan/gutenberg-parser-rs/tree/master/bindings), and notably to those that core Gutenberg cares about: the server, in the form of a PHP extension; the client, via WebAssembly binary and an ASM.js fallback. Through use of the Hoa compiler, native PHP bindings are also available, including an experimental [PCRE parser](\r\nhttps://github.com/Hywan/gutenberg-parser-rs/blob/php_fallback/bindings/php/pure/ParserPCRE.php).\r\n- @dmsnell wrote a hand-coded parser #8083, working with the PHP interpreter to seek optimal performance.\r\n- @pento asked the question of \"if we start with the spec grammar, how much can we gain from fine-tuning it?\" in Improve Parser Performance #8044.\r\n\r\nThese experiments have been benchmarked using `gutenberg-document-library`, and results have been extremely encouraging.\r\n\r\n_to do: insert benchmark tables for CPU and memory_"},"comment":{"url":"https://api.github.com/repos/WordPress/gutenberg/issues/comments/418325471","html_url":"https://github.com/WordPress/gutenberg/issues/8244#issuecomment-418325471","issue_url":"https://api.github.com/repos/WordPress/gutenberg/issues/8244","id":418325471,"node_id":"MDEyOklzc3VlQ29tbWVudDQxODMyNTQ3MQ==","user":{"login":"Hywan","id":946104,"node_id":"MDQ6VXNlcjk0NjEwNA==","avatar_url":"https://avatars2.githubusercontent.com/u/946104?v=4","gravatar_id":"","url":"https://api.github.com/users/Hywan","html_url":"https://github.com/Hywan","followers_url":"https://api.github.com/users/Hywan/followers","following_url":"https://api.github.com/users/Hywan/following{/other_user}","gists_url":"https://api.github.com/users/Hywan/gists{/gist_id}","starred_url":"https://api.github.com/users/Hywan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Hywan/subscriptions","organizations_url":"https://api.github.com/users/Hywan/orgs","repos_url":"https://api.github.com/users/Hywan/repos","events_url":"https://api.github.com/users/Hywan/events{/privacy}","received_events_url":"https://api.github.com/users/Hywan/received_events","type":"User","site_admin":false},"created_at":"2018-09-04T10:59:55Z","updated_at":"2018-09-04T10:59:55Z","author_association":"NONE","body":"Side note: I've found another optimisation for the WASM binding. The average speedup is now 159 instead of 86 :tada:. And I achieved my goal with `moby-dick-parsed.html` which is parsed in 14ms, and `pygmalian-raw-html` in 1ms.\r\n\r\nMy detailed comment has been updated accordingly."}}
{"actor":{"display_login":"Hywan"}}