{"action":"created","issue":{"url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/48","repository_url":"https://api.github.com/repos/benjamin-hodgson/Pidgin","labels_url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/48/labels{/name}","comments_url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/48/comments","events_url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/48/events","html_url":"https://github.com/benjamin-hodgson/Pidgin/issues/48","id":491229602,"node_id":"MDU6SXNzdWU0OTEyMjk2MDI=","number":48,"title":"Resumable Parsing","user":{"login":"TehPers","id":2214247,"node_id":"MDQ6VXNlcjIyMTQyNDc=","avatar_url":"https://avatars1.githubusercontent.com/u/2214247?v=4","gravatar_id":"","url":"https://api.github.com/users/TehPers","html_url":"https://github.com/TehPers","followers_url":"https://api.github.com/users/TehPers/followers","following_url":"https://api.github.com/users/TehPers/following{/other_user}","gists_url":"https://api.github.com/users/TehPers/gists{/gist_id}","starred_url":"https://api.github.com/users/TehPers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TehPers/subscriptions","organizations_url":"https://api.github.com/users/TehPers/orgs","repos_url":"https://api.github.com/users/TehPers/repos","events_url":"https://api.github.com/users/TehPers/events{/privacy}","received_events_url":"https://api.github.com/users/TehPers/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-09-09T17:34:37Z","updated_at":"2019-09-09T17:43:45Z","closed_at":null,"author_association":"NONE","body":"I would love to be able to resume parsing once I finish parsing something. For example, it would be nice to do something like this:\r\n\r\n```cs\r\nIEnumerable<Token> ParseStream(TextReader stream) {\r\n    var result = this.Parser.Parse(stream);\r\n    while (result.Success) {\r\n        yield return result.Value;\r\n        result = result.ParseAgain(); // This would probably be implemented differently\r\n    }\r\n}\r\n```\r\n\r\nThis would allow us to parse from an input stream as needed, without needing to store all the output tokens in memory. For example, we could process a 2GB+ file and write the output into another file without using very much memory.\r\n\r\nSprache has an equivalent to this, but does not support parsing from a stream:\r\n\r\n```cs\r\nIEnumerable<Token> ParseStream(string input) {\r\n    var result = this.Parser.TryParse(input);\r\n    while (result.WasSuccessful) {\r\n        yield return result.Value;\r\n        result = this.Parser(result.Remainder);\r\n    }\r\n}\r\n```\r\n\r\nOne possible approach is to keep a reference to the `ParseState<T>` inside the object that `Parse` returns."},"comment":{"url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/comments/529590115","html_url":"https://github.com/benjamin-hodgson/Pidgin/issues/48#issuecomment-529590115","issue_url":"https://api.github.com/repos/benjamin-hodgson/Pidgin/issues/48","id":529590115,"node_id":"MDEyOklzc3VlQ29tbWVudDUyOTU5MDExNQ==","user":{"login":"benjamin-hodgson","id":3857587,"node_id":"MDQ6VXNlcjM4NTc1ODc=","avatar_url":"https://avatars2.githubusercontent.com/u/3857587?v=4","gravatar_id":"","url":"https://api.github.com/users/benjamin-hodgson","html_url":"https://github.com/benjamin-hodgson","followers_url":"https://api.github.com/users/benjamin-hodgson/followers","following_url":"https://api.github.com/users/benjamin-hodgson/following{/other_user}","gists_url":"https://api.github.com/users/benjamin-hodgson/gists{/gist_id}","starred_url":"https://api.github.com/users/benjamin-hodgson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benjamin-hodgson/subscriptions","organizations_url":"https://api.github.com/users/benjamin-hodgson/orgs","repos_url":"https://api.github.com/users/benjamin-hodgson/repos","events_url":"https://api.github.com/users/benjamin-hodgson/events{/privacy}","received_events_url":"https://api.github.com/users/benjamin-hodgson/received_events","type":"User","site_admin":false},"created_at":"2019-09-09T17:43:45Z","updated_at":"2019-09-09T17:43:45Z","author_association":"OWNER","body":"Does this work?\r\n\r\n```csharp\r\nIEnumerable<Token> ParseStream(TextReader stream)\r\n{\r\n    Result result;\r\n    do\r\n    {\r\n        result = parser.Parse(stream);\r\n        yield return result.Value;\r\n    }\r\n    while (result.Success);\r\n}\r\n```\r\n\r\nAfter the call to `parser.Parse` the stream is left at the position the parser got up to."}}
{"actor":{"display_login":"benjamin-hodgson"}}