{"action":"created","issue":{"url":"https://api.github.com/repos/metaborg/spt/issues/15","repository_url":"https://api.github.com/repos/metaborg/spt","labels_url":"https://api.github.com/repos/metaborg/spt/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/metaborg/spt/issues/15/comments","events_url":"https://api.github.com/repos/metaborg/spt/issues/15/events","html_url":"https://github.com/metaborg/spt/pull/15","id":178544097,"number":15,"title":"Fixes for test extraction","user":{"login":"VolkerL","id":3659862,"avatar_url":"https://avatars.githubusercontent.com/u/3659862?v=3","gravatar_id":"","url":"https://api.github.com/users/VolkerL","html_url":"https://github.com/VolkerL","followers_url":"https://api.github.com/users/VolkerL/followers","following_url":"https://api.github.com/users/VolkerL/following{/other_user}","gists_url":"https://api.github.com/users/VolkerL/gists{/gist_id}","starred_url":"https://api.github.com/users/VolkerL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VolkerL/subscriptions","organizations_url":"https://api.github.com/users/VolkerL/orgs","repos_url":"https://api.github.com/users/VolkerL/repos","events_url":"https://api.github.com/users/VolkerL/events{/privacy}","received_events_url":"https://api.github.com/users/VolkerL/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-09-22T07:50:32Z","updated_at":"2016-09-22T10:59:53Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/metaborg/spt/pulls/15","html_url":"https://github.com/metaborg/spt/pull/15","diff_url":"https://github.com/metaborg/spt/pull/15.diff","patch_url":"https://github.com/metaborg/spt/pull/15.patch"},"body":"This PR offers some fixes for test extraction.\r\nThe bugs were encountered during compiler construction labs, so I tried to make test extraction a bit more robust.\r\n\r\nIt also adds a new language project MiniSQL.\r\nThis language is used in the test suite for SPT, as SPT has to test a language, so to test SPT with SPT, we still need a base language to test.\r\n\r\nThe SPT test suite has been updated.\r\nSPT is now being properly tested with itself.\r\n\r\nAdded the following expectations:\r\n- `analysis succeeds` equivalent to `0 errors`\r\n- `analysis fails` equivalent to `> 0 errors`\r\nI added this syntactic sugar as it is close to the `parse` expectation in syntax and I personally like having them :P"},"comment":{"url":"https://api.github.com/repos/metaborg/spt/issues/comments/248870360","html_url":"https://github.com/metaborg/spt/pull/15#issuecomment-248870360","issue_url":"https://api.github.com/repos/metaborg/spt/issues/15","id":248870360,"user":{"login":"VolkerL","id":3659862,"avatar_url":"https://avatars.githubusercontent.com/u/3659862?v=3","gravatar_id":"","url":"https://api.github.com/users/VolkerL","html_url":"https://github.com/VolkerL","followers_url":"https://api.github.com/users/VolkerL/followers","following_url":"https://api.github.com/users/VolkerL/following{/other_user}","gists_url":"https://api.github.com/users/VolkerL/gists{/gist_id}","starred_url":"https://api.github.com/users/VolkerL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VolkerL/subscriptions","organizations_url":"https://api.github.com/users/VolkerL/orgs","repos_url":"https://api.github.com/users/VolkerL/repos","events_url":"https://api.github.com/users/VolkerL/events{/privacy}","received_events_url":"https://api.github.com/users/VolkerL/received_events","type":"User","site_admin":false},"created_at":"2016-09-22T10:59:53Z","updated_at":"2016-09-22T10:59:53Z","body":"The `<operator?> <int> <severity>` (e.g., `> 0 errors`) expectations are about semantic messages (those caused by the analysis step).\r\nTherefore parse errors can not be tested with them.\r\nSo if parsing fails, but analysis does not produce any errors, `analysis fails` would fail.\r\nIf parsing fails and analysis produces an error, `analysis fails` would succeed.\r\n\r\nHowever, if no parse expectation is provided, SPT inserts a `parse succeeds` expectation automatically.\r\nSo regardless of whether `analysis fails` or `> 0 errors` will succeed, the inserted `parse succeeds` expectation will fail, causing your test to fail.\r\n\r\nSo every test with parse errors fails, unless you explicitly add a `parse fails` expectation.\r\nIn which case it is indeed intended behaviour because the tester indicated that he wanted it to be that way.\r\n"}}
{"actor":{"display_login":"VolkerL"}}