{"action":"created","comment":{"url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/comments/88880082","pull_request_review_id":9438064,"id":88880082,"diff_hunk":"@@ -281,6 +287,65 @@ fn error_test1() {\n }\n \n #[test]\n+fn error_recovery_eof() {\n+    let errors = RefCell::new(vec![]);\n+    util::test(|v| error_recovery::parse_Item(&errors, v), \"--\", '!'.to_string());\n+\n+    assert_eq!(errors.borrow().len(), 1);\n+    assert_eq!(errors.borrow()[0], ParseError::UnrecognizedToken {\n+        token: None,\n+        expected: vec![],\n+    });\n+}\n+\n+#[test]\n+fn error_recovery_eof_without_recovery() {\n+    let errors = RefCell::new(vec![]);\n+    let tokens = util::tok::tokenize(\"-\").into_iter().map(|t| t.1);\n+    let result = error_recovery::parse_Item(&errors, tokens);\n+    assert_eq!(result, Err(ParseError::UnrecognizedToken {\n+        token: None,\n+        expected: vec![],\n+    }));\n+}\n+\n+#[test]\n+fn error_recovery_extra_token() {\n+    let errors = RefCell::new(vec![]);\n+    util::test(|v| error_recovery::parse_Item(&errors, v), \"(++)\", \"()\".to_string());\n+\n+    assert_eq!(errors.borrow().len(), 1);\n+    assert_eq!(errors.borrow()[0], ParseError::UnrecognizedToken {\n+        token: Some(((), Tok::Plus,())),\n+        expected: vec![],\n+    });\n+}\n+\n+#[test]\n+fn error_recovery_dont_drop_unrecognized_token() {\n+    let errors = RefCell::new(vec![]);\n+    util::test(|v| error_recovery::parse_Item(&errors, v), \"(--)\", \"!\".to_string());","path":"lalrpop-test/src/main.rs","position":72,"original_position":72,"commit_id":"e15be2beff6ef4b689bcbbf102c84058139b61ae","original_commit_id":"e15be2beff6ef4b689bcbbf102c84058139b61ae","user":{"login":"Marwes","id":957312,"avatar_url":"https://avatars.githubusercontent.com/u/957312?v=3","gravatar_id":"","url":"https://api.github.com/users/Marwes","html_url":"https://github.com/Marwes","followers_url":"https://api.github.com/users/Marwes/followers","following_url":"https://api.github.com/users/Marwes/following{/other_user}","gists_url":"https://api.github.com/users/Marwes/gists{/gist_id}","starred_url":"https://api.github.com/users/Marwes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Marwes/subscriptions","organizations_url":"https://api.github.com/users/Marwes/orgs","repos_url":"https://api.github.com/users/Marwes/repos","events_url":"https://api.github.com/users/Marwes/events{/privacy}","received_events_url":"https://api.github.com/users/Marwes/received_events","type":"User","site_admin":false},"body":"That's just due to how lazily I implemented the parser https://github.com/nikomatsakis/lalrpop/pull/160/files#diff-a8585f30f0fedc26b330b3798c5c7febR22. I could change it to to wrap the result in parentheses.","created_at":"2016-11-21T11:58:29Z","updated_at":"2016-11-21T11:58:30Z","html_url":"https://github.com/nikomatsakis/lalrpop/pull/160#discussion_r88880082","pull_request_url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160","_links":{"self":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/comments/88880082"},"html":{"href":"https://github.com/nikomatsakis/lalrpop/pull/160#discussion_r88880082"},"pull_request":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160"}}},"pull_request":{"url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160","id":92865095,"html_url":"https://github.com/nikomatsakis/lalrpop/pull/160","diff_url":"https://github.com/nikomatsakis/lalrpop/pull/160.diff","patch_url":"https://github.com/nikomatsakis/lalrpop/pull/160.patch","issue_url":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/160","number":160,"state":"open","locked":false,"title":"feat: Implement error recovery","user":{"login":"Marwes","id":957312,"avatar_url":"https://avatars.githubusercontent.com/u/957312?v=3","gravatar_id":"","url":"https://api.github.com/users/Marwes","html_url":"https://github.com/Marwes","followers_url":"https://api.github.com/users/Marwes/followers","following_url":"https://api.github.com/users/Marwes/following{/other_user}","gists_url":"https://api.github.com/users/Marwes/gists{/gist_id}","starred_url":"https://api.github.com/users/Marwes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Marwes/subscriptions","organizations_url":"https://api.github.com/users/Marwes/orgs","repos_url":"https://api.github.com/users/Marwes/repos","events_url":"https://api.github.com/users/Marwes/events{/privacy}","received_events_url":"https://api.github.com/users/Marwes/received_events","type":"User","site_admin":false},"body":"Work in progress branch which aims to allow parsers to recover from parse errors.\r\n\r\nThis adds the `error` keyword (bikesheddable) to LALRPOPs syntax. If `error` is encountered during parsing the parser goes into recovery mode where it will first pop states until it finds a state which it can potentially recover from. After finding a recoverable state the next step is to skip tokens until a token is found which allows the parser to continue. Recovering from an error acts as a successful parse so it is up to the user to record this error in some way.\r\n\r\n### Unresolved questions\r\n- [x] ~~Is it possible to have recovery for LALR grammars as well? (Need to look into this but haven't found anything concrete yet)~~ I guess his might come for free actually?\r\n\r\n### TODO\r\n- [x] Recover from errors at EOF\r\n- [x] Allow the error recovered from to be inspected in its action `\"+\" <e: error> => { /* use `e` */ }`\r\n- [x] ~~Implement recovery for recursive ascent parsers~~ Skipping this until later when the parse_table implementation has seen some use.\r\n- [x] Implement `unimplemented!()` stubs (example generation)\r\n- [x] Documentation\r\n- [x] Clean up commits\r\n\r\n### Reading\r\nhttps://arxiv.org/pdf/1010.1234.pdf\r\nhttps://www.cs.clemson.edu/course/cpsc827/material/LRk/LR%20Error%20Recovery.pdf\r\nhttps://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=5&ved=0ahUKEwjt9cTt6JjQAhVGjSwKHV0bAigQFgg7MAQ&url=http%3A%2F%2Fwww.cs.princeton.edu%2Fcourses%2Farchive%2Fspr05%2Fcos320%2Fnotes%2F7-parsing-error.ppt&usg=AFQjCNF0I2l3VAMVyT47DIYGv6wZOUzP2A&sig2=0-c3w6nOFevP5nTamen3dg&cad=rja\r\n\r\ncc #156\r\n\r\nBREAKING CHANGE\r\n\r\nThis adds the `error` keyword to grammar definitions.","created_at":"2016-11-08T21:44:09Z","updated_at":"2016-11-21T11:58:30Z","closed_at":null,"merged_at":null,"merge_commit_sha":"858a0896308a537e56e1311b488ef11de577d33a","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160/commits","review_comments_url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160/comments","review_comment_url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/comments{/number}","comments_url":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/160/comments","statuses_url":"https://api.github.com/repos/nikomatsakis/lalrpop/statuses/e15be2beff6ef4b689bcbbf102c84058139b61ae","head":{"label":"Marwes:symbol_error","ref":"symbol_error","sha":"e15be2beff6ef4b689bcbbf102c84058139b61ae","user":{"login":"Marwes","id":957312,"avatar_url":"https://avatars.githubusercontent.com/u/957312?v=3","gravatar_id":"","url":"https://api.github.com/users/Marwes","html_url":"https://github.com/Marwes","followers_url":"https://api.github.com/users/Marwes/followers","following_url":"https://api.github.com/users/Marwes/following{/other_user}","gists_url":"https://api.github.com/users/Marwes/gists{/gist_id}","starred_url":"https://api.github.com/users/Marwes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Marwes/subscriptions","organizations_url":"https://api.github.com/users/Marwes/orgs","repos_url":"https://api.github.com/users/Marwes/repos","events_url":"https://api.github.com/users/Marwes/events{/privacy}","received_events_url":"https://api.github.com/users/Marwes/received_events","type":"User","site_admin":false},"repo":{"id":73131401,"name":"lalrpop","full_name":"Marwes/lalrpop","owner":{"login":"Marwes","id":957312,"avatar_url":"https://avatars.githubusercontent.com/u/957312?v=3","gravatar_id":"","url":"https://api.github.com/users/Marwes","html_url":"https://github.com/Marwes","followers_url":"https://api.github.com/users/Marwes/followers","following_url":"https://api.github.com/users/Marwes/following{/other_user}","gists_url":"https://api.github.com/users/Marwes/gists{/gist_id}","starred_url":"https://api.github.com/users/Marwes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Marwes/subscriptions","organizations_url":"https://api.github.com/users/Marwes/orgs","repos_url":"https://api.github.com/users/Marwes/repos","events_url":"https://api.github.com/users/Marwes/events{/privacy}","received_events_url":"https://api.github.com/users/Marwes/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Marwes/lalrpop","description":"LR(1) parser generator for Rust","fork":true,"url":"https://api.github.com/repos/Marwes/lalrpop","forks_url":"https://api.github.com/repos/Marwes/lalrpop/forks","keys_url":"https://api.github.com/repos/Marwes/lalrpop/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Marwes/lalrpop/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Marwes/lalrpop/teams","hooks_url":"https://api.github.com/repos/Marwes/lalrpop/hooks","issue_events_url":"https://api.github.com/repos/Marwes/lalrpop/issues/events{/number}","events_url":"https://api.github.com/repos/Marwes/lalrpop/events","assignees_url":"https://api.github.com/repos/Marwes/lalrpop/assignees{/user}","branches_url":"https://api.github.com/repos/Marwes/lalrpop/branches{/branch}","tags_url":"https://api.github.com/repos/Marwes/lalrpop/tags","blobs_url":"https://api.github.com/repos/Marwes/lalrpop/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Marwes/lalrpop/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Marwes/lalrpop/git/refs{/sha}","trees_url":"https://api.github.com/repos/Marwes/lalrpop/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Marwes/lalrpop/statuses/{sha}","languages_url":"https://api.github.com/repos/Marwes/lalrpop/languages","stargazers_url":"https://api.github.com/repos/Marwes/lalrpop/stargazers","contributors_url":"https://api.github.com/repos/Marwes/lalrpop/contributors","subscribers_url":"https://api.github.com/repos/Marwes/lalrpop/subscribers","subscription_url":"https://api.github.com/repos/Marwes/lalrpop/subscription","commits_url":"https://api.github.com/repos/Marwes/lalrpop/commits{/sha}","git_commits_url":"https://api.github.com/repos/Marwes/lalrpop/git/commits{/sha}","comments_url":"https://api.github.com/repos/Marwes/lalrpop/comments{/number}","issue_comment_url":"https://api.github.com/repos/Marwes/lalrpop/issues/comments{/number}","contents_url":"https://api.github.com/repos/Marwes/lalrpop/contents/{+path}","compare_url":"https://api.github.com/repos/Marwes/lalrpop/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Marwes/lalrpop/merges","archive_url":"https://api.github.com/repos/Marwes/lalrpop/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Marwes/lalrpop/downloads","issues_url":"https://api.github.com/repos/Marwes/lalrpop/issues{/number}","pulls_url":"https://api.github.com/repos/Marwes/lalrpop/pulls{/number}","milestones_url":"https://api.github.com/repos/Marwes/lalrpop/milestones{/number}","notifications_url":"https://api.github.com/repos/Marwes/lalrpop/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Marwes/lalrpop/labels{/name}","releases_url":"https://api.github.com/repos/Marwes/lalrpop/releases{/id}","deployments_url":"https://api.github.com/repos/Marwes/lalrpop/deployments","created_at":"2016-11-07T23:49:10Z","updated_at":"2016-11-07T23:49:11Z","pushed_at":"2016-11-19T10:35:33Z","git_url":"git://github.com/Marwes/lalrpop.git","ssh_url":"git@github.com:Marwes/lalrpop.git","clone_url":"https://github.com/Marwes/lalrpop.git","svn_url":"https://github.com/Marwes/lalrpop","homepage":"","size":4514,"stargazers_count":0,"watchers_count":0,"language":"Rust","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"nikomatsakis:master","ref":"master","sha":"dba28217c9dec2ad2b105f447b8d5319e7a4f1ca","user":{"login":"nikomatsakis","id":155238,"avatar_url":"https://avatars.githubusercontent.com/u/155238?v=3","gravatar_id":"","url":"https://api.github.com/users/nikomatsakis","html_url":"https://github.com/nikomatsakis","followers_url":"https://api.github.com/users/nikomatsakis/followers","following_url":"https://api.github.com/users/nikomatsakis/following{/other_user}","gists_url":"https://api.github.com/users/nikomatsakis/gists{/gist_id}","starred_url":"https://api.github.com/users/nikomatsakis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nikomatsakis/subscriptions","organizations_url":"https://api.github.com/users/nikomatsakis/orgs","repos_url":"https://api.github.com/users/nikomatsakis/repos","events_url":"https://api.github.com/users/nikomatsakis/events{/privacy}","received_events_url":"https://api.github.com/users/nikomatsakis/received_events","type":"User","site_admin":false},"repo":{"id":37341121,"name":"lalrpop","full_name":"nikomatsakis/lalrpop","owner":{"login":"nikomatsakis","id":155238,"avatar_url":"https://avatars.githubusercontent.com/u/155238?v=3","gravatar_id":"","url":"https://api.github.com/users/nikomatsakis","html_url":"https://github.com/nikomatsakis","followers_url":"https://api.github.com/users/nikomatsakis/followers","following_url":"https://api.github.com/users/nikomatsakis/following{/other_user}","gists_url":"https://api.github.com/users/nikomatsakis/gists{/gist_id}","starred_url":"https://api.github.com/users/nikomatsakis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nikomatsakis/subscriptions","organizations_url":"https://api.github.com/users/nikomatsakis/orgs","repos_url":"https://api.github.com/users/nikomatsakis/repos","events_url":"https://api.github.com/users/nikomatsakis/events{/privacy}","received_events_url":"https://api.github.com/users/nikomatsakis/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/nikomatsakis/lalrpop","description":"LR(1) parser generator for Rust","fork":false,"url":"https://api.github.com/repos/nikomatsakis/lalrpop","forks_url":"https://api.github.com/repos/nikomatsakis/lalrpop/forks","keys_url":"https://api.github.com/repos/nikomatsakis/lalrpop/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nikomatsakis/lalrpop/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nikomatsakis/lalrpop/teams","hooks_url":"https://api.github.com/repos/nikomatsakis/lalrpop/hooks","issue_events_url":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/events{/number}","events_url":"https://api.github.com/repos/nikomatsakis/lalrpop/events","assignees_url":"https://api.github.com/repos/nikomatsakis/lalrpop/assignees{/user}","branches_url":"https://api.github.com/repos/nikomatsakis/lalrpop/branches{/branch}","tags_url":"https://api.github.com/repos/nikomatsakis/lalrpop/tags","blobs_url":"https://api.github.com/repos/nikomatsakis/lalrpop/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nikomatsakis/lalrpop/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nikomatsakis/lalrpop/git/refs{/sha}","trees_url":"https://api.github.com/repos/nikomatsakis/lalrpop/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nikomatsakis/lalrpop/statuses/{sha}","languages_url":"https://api.github.com/repos/nikomatsakis/lalrpop/languages","stargazers_url":"https://api.github.com/repos/nikomatsakis/lalrpop/stargazers","contributors_url":"https://api.github.com/repos/nikomatsakis/lalrpop/contributors","subscribers_url":"https://api.github.com/repos/nikomatsakis/lalrpop/subscribers","subscription_url":"https://api.github.com/repos/nikomatsakis/lalrpop/subscription","commits_url":"https://api.github.com/repos/nikomatsakis/lalrpop/commits{/sha}","git_commits_url":"https://api.github.com/repos/nikomatsakis/lalrpop/git/commits{/sha}","comments_url":"https://api.github.com/repos/nikomatsakis/lalrpop/comments{/number}","issue_comment_url":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/comments{/number}","contents_url":"https://api.github.com/repos/nikomatsakis/lalrpop/contents/{+path}","compare_url":"https://api.github.com/repos/nikomatsakis/lalrpop/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nikomatsakis/lalrpop/merges","archive_url":"https://api.github.com/repos/nikomatsakis/lalrpop/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nikomatsakis/lalrpop/downloads","issues_url":"https://api.github.com/repos/nikomatsakis/lalrpop/issues{/number}","pulls_url":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls{/number}","milestones_url":"https://api.github.com/repos/nikomatsakis/lalrpop/milestones{/number}","notifications_url":"https://api.github.com/repos/nikomatsakis/lalrpop/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nikomatsakis/lalrpop/labels{/name}","releases_url":"https://api.github.com/repos/nikomatsakis/lalrpop/releases{/id}","deployments_url":"https://api.github.com/repos/nikomatsakis/lalrpop/deployments","created_at":"2015-06-12T19:41:37Z","updated_at":"2016-11-17T03:35:17Z","pushed_at":"2016-11-21T10:55:23Z","git_url":"git://github.com/nikomatsakis/lalrpop.git","ssh_url":"git@github.com:nikomatsakis/lalrpop.git","clone_url":"https://github.com/nikomatsakis/lalrpop.git","svn_url":"https://github.com/nikomatsakis/lalrpop","homepage":"","size":4734,"stargazers_count":324,"watchers_count":324,"language":"Rust","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":37,"mirror_url":null,"open_issues_count":52,"forks":37,"open_issues":52,"watchers":324,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160"},"html":{"href":"https://github.com/nikomatsakis/lalrpop/pull/160"},"issue":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/160"},"comments":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/issues/160/comments"},"review_comments":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160/comments"},"review_comment":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/pulls/160/commits"},"statuses":{"href":"https://api.github.com/repos/nikomatsakis/lalrpop/statuses/e15be2beff6ef4b689bcbbf102c84058139b61ae"}}}}
{"actor":{"display_login":"Marwes"}}