{"action":"created","issue":{"url":"https://api.github.com/repos/tweag/asterius/issues/31","repository_url":"https://api.github.com/repos/tweag/asterius","labels_url":"https://api.github.com/repos/tweag/asterius/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/tweag/asterius/issues/31/comments","events_url":"https://api.github.com/repos/tweag/asterius/issues/31/events","html_url":"https://github.com/tweag/asterius/issues/31","id":371674527,"node_id":"MDU6SXNzdWUzNzE2NzQ1Mjc=","number":31,"title":"Investigate hidden binaryen workarounds","user":{"login":"TerrorJack","id":3889585,"node_id":"MDQ6VXNlcjM4ODk1ODU=","avatar_url":"https://avatars3.githubusercontent.com/u/3889585?v=4","gravatar_id":"","url":"https://api.github.com/users/TerrorJack","html_url":"https://github.com/TerrorJack","followers_url":"https://api.github.com/users/TerrorJack/followers","following_url":"https://api.github.com/users/TerrorJack/following{/other_user}","gists_url":"https://api.github.com/users/TerrorJack/gists{/gist_id}","starred_url":"https://api.github.com/users/TerrorJack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TerrorJack/subscriptions","organizations_url":"https://api.github.com/users/TerrorJack/orgs","repos_url":"https://api.github.com/users/TerrorJack/repos","events_url":"https://api.github.com/users/TerrorJack/events{/privacy}","received_events_url":"https://api.github.com/users/TerrorJack/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-10-18T19:21:29Z","updated_at":"2018-10-19T07:15:41Z","closed_at":null,"author_association":"MEMBER","body":"When comparing binary code emitted by binaryen/new backend, I discovered that binaryen doesn't simply do a post-order traversal on the expression IR as expected; there are quite some hidden transformations/workarounds here and there, which are mostly undocumented and there's no way to opt out even when optimize/shrink level is set to 0. A non-comprehensive list including workarounds I discovered so far:\r\n\r\n* When binaryen adds default passes, some are always added regardless of optimize/shrink level\r\n* Not reporting an error on receiving zero alignments, instead, falling back to \"natural\" alignment depending on byte length without a single warning.\r\n* Combining and remapping locals in the code section, so no more than 4 `locals` buckets appear in a function definition. Non-parameter local indices don't correspond to input expression IR (although there does exist a bijection)\r\n* Flattening anonymous blocks.\r\n* Emitting `unreachable` instructions not present in the original IR.\r\n\r\nI've patched binaryen and shrank the list above to only contain the last entry. As long as the last piece is fit into the jigsaw puzzle, we'll base everything on the new backend right away.\r\n\r\nWe're not accusing binaryen's deficiencies here (though the caveats probably should be mentioned in docs); however, this debugging experience does reveal some additional mismatches between what binaryen offers and what we expect:\r\n\r\n* When we pass expression IR to the serializer, we don't expect *ANY* additional magic; any optimization is supposed to be done (and can be done) pre-serialization.\r\n* The linker/debugger performs aggressive IR rewriting and assumes IR isn't twisted in some way we aren't aware of. There is likely some dangerous chemistry between binaryen's rewriting and our own's.\r\n* When we serialize the expression IR, there may be validation errors hiding somewhere, in which case:\r\n    * If validation is only done upon real execution: fine.\r\n    * If a validation pass exists before serialization: find and report them. It's fine to have some false negatives.\r\n    * Status quo: binaryen implements the workaround of inserting the stack-polymorphic `unreachable` instruction in quite some places. Hand it a mal-formed expression IR, it uses this trick to make the IR automagically pass validation...\r\n    * What we expect: If such a trick is *really* needed, it indicates some fundamental flaw in how we engineer the expression IR. If we don't change the IR, at least the trick should be done pre-serialization.\r\n    * Anyway, it's not okay if we thought we generated a well-typed expression but we really didn't, and when we step out the comfort zone binaryen once offered, we're thrown into a chaotic and cold universe."},"comment":{"url":"https://api.github.com/repos/tweag/asterius/issues/comments/431268100","html_url":"https://github.com/tweag/asterius/issues/31#issuecomment-431268100","issue_url":"https://api.github.com/repos/tweag/asterius/issues/31","id":431268100,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMTI2ODEwMA==","user":{"login":"TerrorJack","id":3889585,"node_id":"MDQ6VXNlcjM4ODk1ODU=","avatar_url":"https://avatars3.githubusercontent.com/u/3889585?v=4","gravatar_id":"","url":"https://api.github.com/users/TerrorJack","html_url":"https://github.com/TerrorJack","followers_url":"https://api.github.com/users/TerrorJack/followers","following_url":"https://api.github.com/users/TerrorJack/following{/other_user}","gists_url":"https://api.github.com/users/TerrorJack/gists{/gist_id}","starred_url":"https://api.github.com/users/TerrorJack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TerrorJack/subscriptions","organizations_url":"https://api.github.com/users/TerrorJack/orgs","repos_url":"https://api.github.com/users/TerrorJack/repos","events_url":"https://api.github.com/users/TerrorJack/events{/privacy}","received_events_url":"https://api.github.com/users/TerrorJack/received_events","type":"User","site_admin":false},"created_at":"2018-10-19T07:15:41Z","updated_at":"2018-10-19T07:15:41Z","author_association":"MEMBER","body":"@mboes Other compilers based on binaryen may already be aware of some workarounds and even relying on them, so the switch shall still be inevitable. Still, yes, I'll factor out a short experience summary and \"what to fix\" list as soon as getting out of this minefield. Some should be doc enhancements, other in code."}}
{"actor":{"display_login":"TerrorJack"}}