{"action":"created","issue":{"url":"https://api.github.com/repos/mesonbuild/meson/issues/951","repository_url":"https://api.github.com/repos/mesonbuild/meson","labels_url":"https://api.github.com/repos/mesonbuild/meson/issues/951/labels{/name}","comments_url":"https://api.github.com/repos/mesonbuild/meson/issues/951/comments","events_url":"https://api.github.com/repos/mesonbuild/meson/issues/951/events","html_url":"https://github.com/mesonbuild/meson/pull/951","id":185028271,"number":951,"title":"Fix Ninja rule nondeterminism by sorting ASCIIbetically.","user":{"login":"obilaniu","id":3268344,"avatar_url":"https://avatars.githubusercontent.com/u/3268344?v=3","gravatar_id":"","url":"https://api.github.com/users/obilaniu","html_url":"https://github.com/obilaniu","followers_url":"https://api.github.com/users/obilaniu/followers","following_url":"https://api.github.com/users/obilaniu/following{/other_user}","gists_url":"https://api.github.com/users/obilaniu/gists{/gist_id}","starred_url":"https://api.github.com/users/obilaniu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/obilaniu/subscriptions","organizations_url":"https://api.github.com/users/obilaniu/orgs","repos_url":"https://api.github.com/users/obilaniu/repos","events_url":"https://api.github.com/users/obilaniu/events{/privacy}","received_events_url":"https://api.github.com/users/obilaniu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-10-25T06:18:28Z","updated_at":"2016-10-25T06:49:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/mesonbuild/meson/pulls/951","html_url":"https://github.com/mesonbuild/meson/pull/951","diff_url":"https://github.com/mesonbuild/meson/pull/951.diff","patch_url":"https://github.com/mesonbuild/meson/pull/951.patch"},"body":"The static linker rule (ar csrD $out $in), in particular, seems\r\nsensitive to the order of the elements in $in. Fix this by sorting\r\nusing Python's sorted(), which by default sorts strings:\r\n\r\n1) ASCIIbetically (and thus independently of locale, which is what we\r\nrequire) and\r\n2) Stably.\r\n\r\nThe same problem probably also exists in other backends elsewhere and\r\nprobably other situations as well.\r\n\r\nSetting the environment variable PYTHONHASHSEED=0 also avoids this\r\nproblem, by removing nondeterminism from iterators over unordered data\r\nstructures like Python dicts.\r\n\r\nThis issue arose while I was trying to enforce _reproducible builds_ but could not. I was building something that depended on GoogleTest's `libgtest_main` C++ library, linked statically from `gtest-all.cc` and `gtest_main.cc`, and with 50% probability a side-by-side build would fail. I managed to isolate the problem to `libgtest_main.a` and looked at `build.ninja`; It turns out that the reproduction would fail whenever Build A had the rule\r\n```\r\nbuild 3rdparty/libgtest_main.a: STATIC_LINKER 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest-all.cc.o 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest_main.cc.o\r\n```\r\nwhile Build B had switched `gtest_main.cc.o` and `gtest-all.cc.o`:\r\n```\r\nbuild 3rdparty/libgtest_main.a: STATIC_LINKER 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest_main.cc.o 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest-all.cc.o \r\n```\r\nThis is in spite of `ar csrD` having been invoked with the `D` flag for deterministic handling.\r\n```bash\r\n$ ar csrD 3rdparty/libgtest_main01.a 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest-all.cc.o 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest_main.cc.o\r\n$ ar csrD 3rdparty/libgtest_main10.a 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest_main.cc.o 3rdparty/gtest_main@sta/_longlongpath_googletest_src_gtest-all.cc.o\r\n$ md5sum 3rdparty/\r\ngtest_main@sta/    libgtest_main01.a  libgtest_main10.a  libgtest_main.a    \r\n$ md5sum 3rdparty/*.a\r\n9dc4a95c2962a4c862fd641e67d1e8b9  3rdparty/libgtest_main01.a\r\na0e9d789f91063f458928787169f346d  3rdparty/libgtest_main10.a\r\n9dc4a95c2962a4c862fd641e67d1e8b9  3rdparty/libgtest_main.a\r\n```"},"comment":{"url":"https://api.github.com/repos/mesonbuild/meson/issues/comments/255951509","html_url":"https://github.com/mesonbuild/meson/pull/951#issuecomment-255951509","issue_url":"https://api.github.com/repos/mesonbuild/meson/issues/951","id":255951509,"user":{"login":"obilaniu","id":3268344,"avatar_url":"https://avatars.githubusercontent.com/u/3268344?v=3","gravatar_id":"","url":"https://api.github.com/users/obilaniu","html_url":"https://github.com/obilaniu","followers_url":"https://api.github.com/users/obilaniu/followers","following_url":"https://api.github.com/users/obilaniu/following{/other_user}","gists_url":"https://api.github.com/users/obilaniu/gists{/gist_id}","starred_url":"https://api.github.com/users/obilaniu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/obilaniu/subscriptions","organizations_url":"https://api.github.com/users/obilaniu/orgs","repos_url":"https://api.github.com/users/obilaniu/repos","events_url":"https://api.github.com/users/obilaniu/events{/privacy}","received_events_url":"https://api.github.com/users/obilaniu/received_events","type":"User","site_admin":false},"created_at":"2016-10-25T06:49:18Z","updated_at":"2016-10-25T06:49:18Z","body":"@nirbheek Reproducible builds are end-goals in and of themselves. (See https://reproducible-builds.org for all the advantages). Debian already manages to build 80% of their packages reproducibly.\r\n\r\nI also updated the comment up there in case you've missed it.\r\n\r\nThe fix is really quite simple for the improvement in determinism it brings.\r\n"}}
{"actor":{"display_login":"obilaniu"}}