{"action":"created","issue":{"url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/360","repository_url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts","labels_url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/360/labels{/name}","comments_url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/360/comments","events_url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/360/events","html_url":"https://github.com/tunnelvisionlabs/antlr4ts/issues/360","id":321683279,"number":360,"title":"Using ECMAScript generators for parse functions","user":{"login":"BurtHarris","id":2715647,"avatar_url":"https://avatars0.githubusercontent.com/u/2715647?v=4","gravatar_id":"","url":"https://api.github.com/users/BurtHarris","html_url":"https://github.com/BurtHarris","followers_url":"https://api.github.com/users/BurtHarris/followers","following_url":"https://api.github.com/users/BurtHarris/following{/other_user}","gists_url":"https://api.github.com/users/BurtHarris/gists{/gist_id}","starred_url":"https://api.github.com/users/BurtHarris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BurtHarris/subscriptions","organizations_url":"https://api.github.com/users/BurtHarris/orgs","repos_url":"https://api.github.com/users/BurtHarris/repos","events_url":"https://api.github.com/users/BurtHarris/events{/privacy}","received_events_url":"https://api.github.com/users/BurtHarris/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-05-09T18:46:44Z","updated_at":"2018-05-16T08:09:07Z","closed_at":null,"author_association":"COLLABORATOR","body":"TypeScript/ECMAScript [generators ](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator ) can be used for a asynchronous parser based on coroutines without much overhead.    The advantage of this is that it disconnects us from the Java stream mechanism, which was designed for environments supporting multiple threads, and allows native NodeJS streams to be used to supply input to a parser.    (JavaScript doesn't in general support threads, and blocking a thread in NodeJS or a browser is bad form.)\r\n\r\nFor example, the hildjj/node-cbor decoder uses (an embedded version of) **binary-parse-stream** to make the parser able to function as a Node.js [`stream.transform`](https://nodejs.org/api/stream.html#stream_class_stream_transform) stream, and support the stream [`pipe()`](https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options) operation, which is the preferred mechanism to implement for file manipulation tools for the Node platform.\r\n\r\nWhile this would possibly require a change the antler4ts version 0 API, it seems to me that it wouldn't be conceptually hard to deal with, and it does seem to be able to be used in a synchronous mode, for those situation where the entire input stream is available at the start of parse/tokenize operations.\r\n\r\nGenerators can use the `yield` expression to act as co-routines and suspend their operation.   Specifically in the model I contemplate.   In the model I contemplate, `yield` would be used to indicate that a grammar requires more input than currently available, and the production would be suspended until another `Buffer` of information was available.    \r\n\r\nFor convenience in the synchronous case, the generated recognizer class could have static functions for top-level productions added which are optimized for the case where the full parse input is available (as either a string or Buffer.)   These would throw if the input wasn't complete according to the rules of the production, and `yield` would never be called.\r\n\r\nAny thoughts/comments on this would be appreciated.  @sharwell, @mike-lischke?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/comments/389433070","html_url":"https://github.com/tunnelvisionlabs/antlr4ts/issues/360#issuecomment-389433070","issue_url":"https://api.github.com/repos/tunnelvisionlabs/antlr4ts/issues/360","id":389433070,"user":{"login":"BurtHarris","id":2715647,"avatar_url":"https://avatars0.githubusercontent.com/u/2715647?v=4","gravatar_id":"","url":"https://api.github.com/users/BurtHarris","html_url":"https://github.com/BurtHarris","followers_url":"https://api.github.com/users/BurtHarris/followers","following_url":"https://api.github.com/users/BurtHarris/following{/other_user}","gists_url":"https://api.github.com/users/BurtHarris/gists{/gist_id}","starred_url":"https://api.github.com/users/BurtHarris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BurtHarris/subscriptions","organizations_url":"https://api.github.com/users/BurtHarris/orgs","repos_url":"https://api.github.com/users/BurtHarris/repos","events_url":"https://api.github.com/users/BurtHarris/events{/privacy}","received_events_url":"https://api.github.com/users/BurtHarris/received_events","type":"User","site_admin":false},"created_at":"2018-05-16T08:09:07Z","updated_at":"2018-05-16T08:09:07Z","author_association":"COLLABORATOR","body":"Just noticed [tsouza/yajs ](https://github.com/tsouza/yajs/wiki/Getting-Started)which appears to be looking for something along the lines I'm proposing.   The author has a dependency on antlr4ts, but it doesn't look like it's been integrated."}}
{"actor":{"display_login":"BurtHarris"}}