{"action":"created","issue":{"url":"https://api.github.com/repos/meteor/meteor/issues/6058","repository_url":"https://api.github.com/repos/meteor/meteor","labels_url":"https://api.github.com/repos/meteor/meteor/issues/6058/labels{/name}","comments_url":"https://api.github.com/repos/meteor/meteor/issues/6058/comments","events_url":"https://api.github.com/repos/meteor/meteor/issues/6058/events","html_url":"https://github.com/meteor/meteor/issues/6058","id":127997191,"number":6058,"title":"Build tool performance improvements","user":{"login":"dgreensp","id":155814,"avatar_url":"https://avatars.githubusercontent.com/u/155814?v=3","gravatar_id":"","url":"https://api.github.com/users/dgreensp","html_url":"https://github.com/dgreensp","followers_url":"https://api.github.com/users/dgreensp/followers","following_url":"https://api.github.com/users/dgreensp/following{/other_user}","gists_url":"https://api.github.com/users/dgreensp/gists{/gist_id}","starred_url":"https://api.github.com/users/dgreensp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgreensp/subscriptions","organizations_url":"https://api.github.com/users/dgreensp/orgs","repos_url":"https://api.github.com/users/dgreensp/repos","events_url":"https://api.github.com/users/dgreensp/events{/privacy}","received_events_url":"https://api.github.com/users/dgreensp/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/meteor/meteor/labels/Project:Isobuild","name":"Project:Isobuild","color":"f7c6c7"},{"url":"https://api.github.com/repos/meteor/meteor/labels/Project:Isobuild:Performance","name":"Project:Isobuild:Performance","color":"fbca04"}],"state":"open","locked":false,"assignee":{"login":"dgreensp","id":155814,"avatar_url":"https://avatars.githubusercontent.com/u/155814?v=3","gravatar_id":"","url":"https://api.github.com/users/dgreensp","html_url":"https://github.com/dgreensp","followers_url":"https://api.github.com/users/dgreensp/followers","following_url":"https://api.github.com/users/dgreensp/following{/other_user}","gists_url":"https://api.github.com/users/dgreensp/gists{/gist_id}","starred_url":"https://api.github.com/users/dgreensp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgreensp/subscriptions","organizations_url":"https://api.github.com/users/dgreensp/orgs","repos_url":"https://api.github.com/users/dgreensp/repos","events_url":"https://api.github.com/users/dgreensp/events{/privacy}","received_events_url":"https://api.github.com/users/dgreensp/received_events","type":"User","site_admin":false},"milestone":null,"comments":16,"created_at":"2016-01-21T19:08:13Z","updated_at":"2016-02-06T00:48:39Z","closed_at":null,"body":"**See also this doc:** https://github.com/meteor/meteor/blob/release-1.3/tools/PERFORMANCE.md\r\n\r\nThis issue tracks build tool performance improvements being worked on in the context of Meteor 1.3.\r\n\r\nWhen talking about build tool performance, the following factors should be taken into account:\r\n\r\n* Initial build vs. rebuild: As you'd expect, rebuild takes much less time than initial build, because of caching.  A performance improvement that cuts down initial build time may not affect rebuild time.  Also, there's a distinction between a client-only and a client/server rebuild.  In addition, the _first_ rebuild after starting an app is currently slower than subsequent rebuilds because of how the constraint solver does caching.\r\n\r\n* Running from release vs. checkout: Most people run Meteor \"from a release,\" meaning they run the installed `meteor` on their machine (with or without `--release`).  When developing and testing Meteor, we often from \"from checkout,\" meaning we run `./meteor` from the checked-out repo.  These two modes have very different performance characteristics.  When you run from checkout, the core packages are loaded from the repo.  This means that, for example, a lot more files are watched, but the constraint solver doesn't have to do nearly as much work because the core package versions are pinned.  While it's easier to debug and inspect Meteor when running from checkout, when working on performance improvements we should focus on the experience of running from release.\r\n\r\n* Size and complexity of app:  One app may be slower to build than another because it is larger or more complex on in any number of dimensions, such as the amount of code to compile or CSS to minify; the compiler plugins used; the size of the dependency graph; or the size in bytes of the packages used.  We should make sure \"skeleton\" (the empty app) is fast, and also that typical \"real apps\" are fast.\r\n\r\n* Hardware and OS:  Numbers will vary based on your CPU speed and disk speed (especially whether you have an SSD or a spinning disk).  Also, the Windows file system has different characteristics from a Mac or Linux file system.\r\n\r\nEven though absolute time measurements will vary from computer to computer and app to app, it's useful to talk in rough numbers about where time is going.\r\n\r\n#### Overview of Rebuilding\r\n\r\nWhen a \"real app\" rebuilds running from release, it may take as much as 10-20 seconds on `1.3-modules-beta.4` (the latest release as of the beginning of this work).  Most of this time goes to \"bundling.\"  The bundler is instrumented using Meteor's profiling library, so if you run `METEOR_PROFILE=1 meteor` in a shell, you'll get a decent overview of where the time is going (under the heading \"Rebuild App\").\r\n\r\nBesides bundling, there are a couple other \"hidden\" sources of rebuild time which you won't see in the profile, but they are small in comparison:\r\n* App start-up (~1 second): Meteor runs two Node processes, one for the CLI tool and one for the app.  On rebuild, the app is killed and restarted, and the message `=> Meteor server restarted` is displayed once the new app process is live and serving.\r\n* File watcher set-up (hundreds of ms): On rebuild, the tool tears down the old file watchers and sets up new ones.  This set-up is performed in parallel with app start-up, so it is not really noticeable when running from release.  (When running from checkout, there are so many files to watch that it may take several seconds and becomes noticeable.)\r\n\r\n#### Potential improvements\r\n\r\nThese are opportunities for half-second to multi-second improvements:\r\n\r\nRebuild:\r\n- [X] Fix linker's broken RAM cache\r\n- [X] Cache code analysis to detect `import` in modules\r\n- [X] Don't stat when writing files\r\n- [X] Upgrade `source-map` NPM package\r\n- [x] Speed up read/stat of NPM dependencies\r\n- [ ] Speed up CSS analysis and concatenation - see https://github.com/meteor/meteor/pull/5875\r\n- [x] Change line-number comments in linked files to not inflate file sizes so much (2-3 times?)\r\n- [x] Overwrite existing bundle on build (for far less file writing)\r\n- [ ] Further investigate source map performance\r\n\r\nInitial build:\r\n- [ ] \"Updating package catalog…\" seems to take a long time; why? (~5 seconds?)\r\n- [x] Implement constraint solver \"fast path\" if `.meteor/versions` is valid (5-8 seconds)\r\n- [x] Optimize package version parsing/sorting when loading catalog data (1-2 seconds out of the above?)\r\n\r\n- [ ] To reproduce: Rebuild gradually gets slower over time and becomes more like 30+ seconds?"},"comment":{"url":"https://api.github.com/repos/meteor/meteor/issues/comments/180638744","html_url":"https://github.com/meteor/meteor/issues/6058#issuecomment-180638744","issue_url":"https://api.github.com/repos/meteor/meteor/issues/6058","id":180638744,"user":{"login":"dgreensp","id":155814,"avatar_url":"https://avatars.githubusercontent.com/u/155814?v=3","gravatar_id":"","url":"https://api.github.com/users/dgreensp","html_url":"https://github.com/dgreensp","followers_url":"https://api.github.com/users/dgreensp/followers","following_url":"https://api.github.com/users/dgreensp/following{/other_user}","gists_url":"https://api.github.com/users/dgreensp/gists{/gist_id}","starred_url":"https://api.github.com/users/dgreensp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgreensp/subscriptions","organizations_url":"https://api.github.com/users/dgreensp/orgs","repos_url":"https://api.github.com/users/dgreensp/repos","events_url":"https://api.github.com/users/dgreensp/events{/privacy}","received_events_url":"https://api.github.com/users/dgreensp/received_events","type":"User","site_admin":false},"created_at":"2016-02-06T00:48:39Z","updated_at":"2016-02-06T00:48:39Z","body":"see also: https://github.com/meteor/meteor/blob/release-1.3/tools/PERFORMANCE.md"}}