{"action":"created","issue":{"url":"https://api.github.com/repos/scala/scala/issues/6314","repository_url":"https://api.github.com/repos/scala/scala","labels_url":"https://api.github.com/repos/scala/scala/issues/6314/labels{/name}","comments_url":"https://api.github.com/repos/scala/scala/issues/6314/comments","events_url":"https://api.github.com/repos/scala/scala/issues/6314/events","html_url":"https://github.com/scala/scala/pull/6314","id":295326849,"number":6314,"title":"Cache classloaders for compiler plugins and macros","user":{"login":"jvican","id":2462974,"avatar_url":"https://avatars0.githubusercontent.com/u/2462974?v=4","gravatar_id":"","url":"https://api.github.com/users/jvican","html_url":"https://github.com/jvican","followers_url":"https://api.github.com/users/jvican/followers","following_url":"https://api.github.com/users/jvican/following{/other_user}","gists_url":"https://api.github.com/users/jvican/gists{/gist_id}","starred_url":"https://api.github.com/users/jvican/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jvican/subscriptions","organizations_url":"https://api.github.com/users/jvican/orgs","repos_url":"https://api.github.com/users/jvican/repos","events_url":"https://api.github.com/users/jvican/events{/privacy}","received_events_url":"https://api.github.com/users/jvican/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/scala/scala/milestones/68","html_url":"https://github.com/scala/scala/milestone/68","labels_url":"https://api.github.com/repos/scala/scala/milestones/68/labels","id":2719693,"number":68,"title":"2.12.5","description":"Merge to 2.12.x. For feature development, please target 2.13.x. 2.12 is now restricted to bug fixes.","creator":{"login":"adriaanm","id":91083,"avatar_url":"https://avatars2.githubusercontent.com/u/91083?v=4","gravatar_id":"","url":"https://api.github.com/users/adriaanm","html_url":"https://github.com/adriaanm","followers_url":"https://api.github.com/users/adriaanm/followers","following_url":"https://api.github.com/users/adriaanm/following{/other_user}","gists_url":"https://api.github.com/users/adriaanm/gists{/gist_id}","starred_url":"https://api.github.com/users/adriaanm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adriaanm/subscriptions","organizations_url":"https://api.github.com/users/adriaanm/orgs","repos_url":"https://api.github.com/users/adriaanm/repos","events_url":"https://api.github.com/users/adriaanm/events{/privacy}","received_events_url":"https://api.github.com/users/adriaanm/received_events","type":"User","site_admin":false},"open_issues":47,"closed_issues":79,"state":"open","created_at":"2017-08-23T23:46:19Z","updated_at":"2018-02-07T23:08:27Z","due_on":"2018-03-02T08:00:00Z","closed_at":null},"comments":5,"created_at":"2018-02-07T23:08:25Z","updated_at":"2018-02-08T11:01:44Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/scala/scala/pulls/6314","html_url":"https://github.com/scala/scala/pull/6314","diff_url":"https://github.com/scala/scala/pull/6314.diff","patch_url":"https://github.com/scala/scala/pull/6314.patch"},"body":"For the rationale and a detailed performance analysis, please check [this\r\nissue](https://github.com/scala/scala-dev/issues/458).\r\n\r\nThis pull request continues the work started by @retronym on caching the\r\nclasspath, and extends it to caching compiler plugins classloaders. In short,\r\nthis change seems to yield compile-time performance improvements ranging from\r\n16% to 35% for projects using compiler plugins.\r\n\r\nThe approach taken in this pull request is to cache only compiler plugins'\r\nclasspaths that contain zip or jar files (to avoid issues with validation). The\r\napproach taken here is a little bit naive and I hope it improves in a follow-up\r\nPR.\r\n\r\nThere is one missing item that I'll add to this pull request during the next\r\ndays: making the cache invalidation algorithm take into account the size of the\r\nclasspath entries to make the cache heuristics more robust.\r\n\r\nHowever, as discussed in https://github.com/scala/scala-dev/issues/458, these\r\nheuristics do not work in scenarios where the timestamp is zero'd and the size\r\ndoes not change (this happens with tools that focus on reproducibility like\r\nPants and Bazel).\r\n\r\nThere is only one way we can fix this issue without relying on heuristics:\r\nhashing the contents of the classpath. However, this is an expensive\r\ncomputation for big classpaths and we want to probably stay away from it.\r\n\r\nTo solve this issue in a friendlier manner, I propose to add a compiler flag so\r\nthat the build tool can share the checksums of every jar they have with the\r\ncompiler and let the compiler invalidate the caches based on those. If this\r\nsounds like a good solution to you, I will open another pull request\r\nimplementing it.\r\n\r\nI'd like this change to target 2.12.5 if possible."},"comment":{"url":"https://api.github.com/repos/scala/scala/issues/comments/364077677","html_url":"https://github.com/scala/scala/pull/6314#issuecomment-364077677","issue_url":"https://api.github.com/repos/scala/scala/issues/6314","id":364077677,"user":{"login":"jvican","id":2462974,"avatar_url":"https://avatars0.githubusercontent.com/u/2462974?v=4","gravatar_id":"","url":"https://api.github.com/users/jvican","html_url":"https://github.com/jvican","followers_url":"https://api.github.com/users/jvican/followers","following_url":"https://api.github.com/users/jvican/following{/other_user}","gists_url":"https://api.github.com/users/jvican/gists{/gist_id}","starred_url":"https://api.github.com/users/jvican/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jvican/subscriptions","organizations_url":"https://api.github.com/users/jvican/orgs","repos_url":"https://api.github.com/users/jvican/repos","events_url":"https://api.github.com/users/jvican/events{/privacy}","received_events_url":"https://api.github.com/users/jvican/received_events","type":"User","site_admin":false},"created_at":"2018-02-08T11:01:44Z","updated_at":"2018-02-08T11:01:44Z","author_association":"MEMBER","body":"@mkeskells Yeah, but remember that `fileKey` keeps being the same even if the contents of the file change.\r\n  \r\n> we can aid the testing in this and provide some feedback as to the impact in our environment\r\n  \r\nI would be very much interested in knowing how this patch affects your environment.\r\n\r\n> Instead of hashing the jar we could use the hashes in the zip entries\r\n  \r\n(Those are erased by Bazel too, to the best of my understanding. All metadata in the zip entries is wiped down.)\r\n\r\n@lrytz I proposed the same fix for an sbt/zinc regression back in the day: https://github.com/sbt/zinc/issues/433. We can have that too, but I feel that we should tackle the issue at the root of the problem: let the build tool, which knows the hashes for every classpath entry, communicate that to the compiler. This way we can reuse the hashes for other tools like Zinc or even third-party compiler plugins. In case build tools do not provide the hashes, we can fallback on the heuristics."}}
{"actor":{"display_login":"jvican"}}