{"action":"closed","issue":{"url":"https://api.github.com/repos/bazelbuild/rules_scala/issues/53","repository_url":"https://api.github.com/repos/bazelbuild/rules_scala","labels_url":"https://api.github.com/repos/bazelbuild/rules_scala/issues/53/labels{/name}","comments_url":"https://api.github.com/repos/bazelbuild/rules_scala/issues/53/comments","events_url":"https://api.github.com/repos/bazelbuild/rules_scala/issues/53/events","html_url":"https://github.com/bazelbuild/rules_scala/issues/53","id":155598031,"number":53,"title":"md5sum hashes do not match when you build the same code twice on macos","user":{"login":"xynny","id":13421989,"avatar_url":"https://avatars.githubusercontent.com/u/13421989?v=3","gravatar_id":"","url":"https://api.github.com/users/xynny","html_url":"https://github.com/xynny","followers_url":"https://api.github.com/users/xynny/followers","following_url":"https://api.github.com/users/xynny/following{/other_user}","gists_url":"https://api.github.com/users/xynny/gists{/gist_id}","starred_url":"https://api.github.com/users/xynny/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xynny/subscriptions","organizations_url":"https://api.github.com/users/xynny/orgs","repos_url":"https://api.github.com/users/xynny/repos","events_url":"https://api.github.com/users/xynny/events{/privacy}","received_events_url":"https://api.github.com/users/xynny/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-05-18T20:49:02Z","updated_at":"2016-05-27T23:48:09Z","closed_at":"2016-05-27T23:48:09Z","body":"Just discovered this recently since I was working on build caching, but it seems like the default scala rule here do not  build  jars with the same hash on two separate runs with  the same inputs.  This seems to break the caching code in bazel that gets a digest with md5. \r\nThis is what I ran on this repo here:\r\n```\r\nbazel build test/... \r\nmd5 bazel-bin/test/*.jar > hash\r\nbazel clean\r\nbazel build test/... \r\nmd5 bazel-bin/test/*.jar > hash2\r\n```\r\n\r\nWhen you diff the two files it shows that all the jars have different hashes.  even the ijars. \r\n\r\n```\r\nXins-MacBook-Pro:rules_scala xinlu$ diff hash hash2\r\n1,3c1,3\r\n< MD5 (bazel-bin/test/ExportOnly_deploy.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/ExportOnly_ijar.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/Exported_deploy.jar) = 4e1b1b620a002219a2edda47dc9a9cd2\r\n---\r\n> MD5 (bazel-bin/test/ExportOnly_deploy.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/ExportOnly_ijar.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/Exported_deploy.jar) = 31ea9c77a3bbf21d2b0ffc6def3247e4\r\n5,6c5,6\r\n< MD5 (bazel-bin/test/HelloLibTest_deploy.jar) = 1faf15bba18d60a4d9fee7937f3d18d6\r\n< MD5 (bazel-bin/test/HelloLib_deploy.jar) = d72085331bdb480df424be08c0b8b5bd\r\n---\r\n> MD5 (bazel-bin/test/HelloLibTest_deploy.jar) = 9822c828fc3a1cbb87dbcd3800ba9e62\r\n> MD5 (bazel-bin/test/HelloLib_deploy.jar) = 398401f2f02554b2d60aa62288cb255f\r\n10,11c10,11\r\n< MD5 (bazel-bin/test/MacroTest_deploy.jar) = 8c4767eead2843d79b850e56c86ea42e\r\n< MD5 (bazel-bin/test/OtherLib_deploy.jar) = 2fc708e492140ed78ff0a489df106dcd\r\n---\r\n> MD5 (bazel-bin/test/MacroTest_deploy.jar) = 8024d3769157936d4777fe21aa88c25e\r\n> MD5 (bazel-bin/test/OtherLib_deploy.jar) = be1812ca6d4eda50fdbb0bf340b088c8\r\n13c13\r\n< MD5 (bazel-bin/test/Runtime_deploy.jar) = fffa4662d3403c7a2daa042154e03f8f\r\n---\r\n> MD5 (bazel-bin/test/Runtime_deploy.jar) = efea7e4838f6bcd8582a0a5d86ca38f8\r\n15,17c15,17\r\n< MD5 (bazel-bin/test/ScalaBinary_deploy.jar) = 75871f3e5ef7addcbbfad04fe813b87a\r\n< MD5 (bazel-bin/test/ScalaLibBinary_deploy.jar) = c46c0b69cdea700f37b40ceeb6b0bdd6\r\n< MD5 (bazel-bin/test/ScalaLibResources_deploy.jar) = 4c0d57f5ad553e91f83f8b8bc866a195\r\n---\r\n> MD5 (bazel-bin/test/ScalaBinary_deploy.jar) = 4d9d183d54111532a7b88067d6e4572d\r\n> MD5 (bazel-bin/test/ScalaLibBinary_deploy.jar) = 79684af04f1f3fa6b581775611b05aac\r\n> MD5 (bazel-bin/test/ScalaLibResources_deploy.jar) = ea4157e3f8f2e59587bddbd26999bd89\r\n19c19\r\n< MD5 (bazel-bin/test/a_deploy.jar) = 08fe25daf5678fe3e1d43bc52c88daf5\r\n---\r\n> MD5 (bazel-bin/test/a_deploy.jar) = 5f071b2165bd91ff1fee97b5409cf282\r\n21,25c21,25\r\n< MD5 (bazel-bin/test/b_deploy.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/b_ijar.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/c_deploy.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/c_ijar.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/d_deploy.jar) = ecb50527b59a959320496d20809276bc\r\n---\r\n> MD5 (bazel-bin/test/b_deploy.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/b_ijar.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/c_deploy.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/c_ijar.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/d_deploy.jar) = 9bbe26c787071564c34af4cd418f7b9d\r\n27,28c27,28\r\n< MD5 (bazel-bin/test/jar_export_deploy.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n< MD5 (bazel-bin/test/jar_export_ijar.jar) = e513f71cb2024bf2a5c6ac0a0208fb5e\r\n---\r\n> MD5 (bazel-bin/test/jar_export_deploy.jar) = 764d68cad512ea85baf75aec623c465e\r\n> MD5 (bazel-bin/test/jar_export_ijar.jar) = 764d68cad512ea85baf75aec623c465e\r\n\r\n---\r\n\r\n```\r\nI think this is because the jar command used by the scala.bzl rule changes the timestamp of the Manifest file even if you touch it. Example:\r\n```\r\n\r\nXins-MacBook-Pro:rules_scala xinlu$ unzip -l bazel-bin/test/ExportOnly_deploy.jar\r\nArchive:  bazel-bin/test/ExportOnly_deploy.jar\r\n  Length     Date   Time    Name\r\n --------    ----   ----    ----\r\n        0  05-18-16 13:58   META-INF/\r\n      118  05-18-16 13:58   META-INF/MANIFEST.MF\r\n --------                   -------\r\n      118                   2 files\r\n```\r\n\r\nBasically this means that the touch to set the timestamp to 1980 didn't matter.  \r\n In our own internal version I changed the jarring to zip -X -q  -FS since -FS keeps timestamps and now the hash is the same between two different builds that have the same inputs because the timestamps match. \r\n\r\nAlso this seems to be on mac, and not linux.  The hashes seem to be okay on linux, but a lot of our devs use macs and it actually breaks their local cache.  "}}
{"actor":{"display_login":"johnynek"}}