{"action":"created","issue":{"url":"https://api.github.com/repos/webpack/webpack/issues/6613","repository_url":"https://api.github.com/repos/webpack/webpack","labels_url":"https://api.github.com/repos/webpack/webpack/issues/6613/labels{/name}","comments_url":"https://api.github.com/repos/webpack/webpack/issues/6613/comments","events_url":"https://api.github.com/repos/webpack/webpack/issues/6613/events","html_url":"https://github.com/webpack/webpack/pull/6613","id":300852120,"number":6613,"title":"improve performance of ModuleConcatenationPlugin for loop","user":{"login":"brentwilton","id":11993233,"avatar_url":"https://avatars2.githubusercontent.com/u/11993233?v=4","gravatar_id":"","url":"https://api.github.com/users/brentwilton","html_url":"https://github.com/brentwilton","followers_url":"https://api.github.com/users/brentwilton/followers","following_url":"https://api.github.com/users/brentwilton/following{/other_user}","gists_url":"https://api.github.com/users/brentwilton/gists{/gist_id}","starred_url":"https://api.github.com/users/brentwilton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brentwilton/subscriptions","organizations_url":"https://api.github.com/users/brentwilton/orgs","repos_url":"https://api.github.com/users/brentwilton/repos","events_url":"https://api.github.com/users/brentwilton/events{/privacy}","received_events_url":"https://api.github.com/users/brentwilton/received_events","type":"User","site_admin":false},"labels":[{"id":572709861,"url":"https://api.github.com/repos/webpack/webpack/labels/PR:%20CI-not-ok","name":"PR: CI-not-ok","color":"d93f0b","default":false},{"id":572817412,"url":"https://api.github.com/repos/webpack/webpack/labels/PR:%20review-outdated","name":"PR: review-outdated","color":"d93f0b","default":false},{"id":575748502,"url":"https://api.github.com/repos/webpack/webpack/labels/PR:%20small","name":"PR: small","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2018-02-28T01:02:04Z","updated_at":"2018-03-02T01:37:52Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/webpack/webpack/pulls/6613","html_url":"https://github.com/webpack/webpack/pull/6613","diff_url":"https://github.com/webpack/webpack/pull/6613.diff","patch_url":"https://github.com/webpack/webpack/pull/6613.patch"},"body":"**What kind of change does this PR introduce?**\r\n\r\nbugfix / performance improvement\r\n\r\n**Did you add tests for your changes?**\r\n\r\nNo\r\n\r\n**If relevant, link to documentation update:**\r\n\r\nN/A\r\n\r\n**Summary**\r\n\r\nIn our large Angular 5 project (~3400 modules) ModuleConcatenationPlugin is now taking 60% of the build time in webpack 4.\r\n\r\nWebpack is spending 869499ms in 'chunk modules optimization' within ModuleConcatenationPlugin.\r\n\r\nMost of this time is consumed in 2 nested for-of loops in the ModuleConcatenationPlugin. By changing the for-of loops to standard for loops we have reduced the time we spend in ModuleConcatenationPlugin from 14 minutes 29 seconds to 2 minutes 35 seconds.\r\n\r\n**Does this PR introduce a breaking change?**\r\n\r\nNo\r\n\r\n**Other information**\r\n\r\nNode: 8.9.3\r\nWebpack: 4.0.1 and 4.0.0\r\nWindows 10\r\n\r\nDuring our prod build the inner for-of loop is visited a total of 7,309,580,444 (7.3 billion) times.\r\n\r\nWhen debugging most of the time was spent in the following 2 for loops in ModuleConcatenationPlugin.js. Original time taken: 14m 29s (869499ms)\r\n\r\nhttps://github.com/webpack/webpack/blob/6a5d081f294a225d9f02b56c55fdcd61b7b4db20/lib/optimize/ModuleConcatenationPlugin.js#L284-L290\r\n\r\nChanging the above for of loop to a standard for loop reduces the time taken to  6m 8s (368453ms)\r\n```\r\nfor (let i = 0; i < newModule.dependencies.length; i++) {\r\n    let dep = newModule.dependencies[i];\r\n    if (dep.module) {\r\n        for (let j = 0; j < dep.module.reasons.length; j++) {\r\n            let reason = dep.module.reasons[j];\r\n            if (reason.dependency === dep) reason.module = newModule;\r\n        }\r\n    }\r\n}\r\n```\r\nMaking a further change to `let reasons = dep.module.reasons;` reduces the time taken to 2m 35s (141326ms)\r\n\r\n```\r\nfor (let i = 0; i < newModule.dependencies.length; i++) {\r\n    let dep = newModule.dependencies[i];\r\n    if (dep.module) {\r\n        let reasons = dep.module.reasons;\r\n        for (let j = 0; j < reasons.length; j++) {\r\n            let reason = reasons[j];\r\n            if (reason.dependency === dep) reason.module = newModule;\r\n        }\r\n    }\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/webpack/webpack/issues/comments/369791420","html_url":"https://github.com/webpack/webpack/pull/6613#issuecomment-369791420","issue_url":"https://api.github.com/repos/webpack/webpack/issues/6613","id":369791420,"user":{"login":"webpack-bot","id":26676291,"avatar_url":"https://avatars0.githubusercontent.com/u/26676291?v=4","gravatar_id":"","url":"https://api.github.com/users/webpack-bot","html_url":"https://github.com/webpack-bot","followers_url":"https://api.github.com/users/webpack-bot/followers","following_url":"https://api.github.com/users/webpack-bot/following{/other_user}","gists_url":"https://api.github.com/users/webpack-bot/gists{/gist_id}","starred_url":"https://api.github.com/users/webpack-bot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/webpack-bot/subscriptions","organizations_url":"https://api.github.com/users/webpack-bot/orgs","repos_url":"https://api.github.com/users/webpack-bot/repos","events_url":"https://api.github.com/users/webpack-bot/events{/privacy}","received_events_url":"https://api.github.com/users/webpack-bot/received_events","type":"User","site_admin":false},"created_at":"2018-03-02T01:37:52Z","updated_at":"2018-03-02T01:37:52Z","author_association":"NONE","body":"<!-- identifier: ci-result -->\n\n@brentwilton Please review the following output log for errors:\n\n``` text\nWorker information [...]\nmode of ‘/usr/local/clang-5.0.0/bin’ changed from 0777 (rwxrwxrwx) to 0775 (rwxrwxr-x)\nBuild system information [...]\nremoved ‘/etc/apt/sources.list.d/basho_riak.list’\n127.0.0.1\tlocalhost\n::1\t ip6-localhost ip6-loopback\nfe00::0\tip6-localnet\nff00::0\tip6-mcastprefix\nff02::1\tip6-allnodes\nff02::2\tip6-allrouters\n172.17.0.21\ttravis-job-webpack-webpack-348045700.travisci.net travis-job-webpack-webpack-348045700\nW: http://ppa.launchpad.net/couchdb/stable/ubuntu/dists/trusty/Release.gpg: Signature by key 15866BAFD9BCC4F3C1E0DFC7D69548E1C17EAB57 uses weak digest algorithm (SHA1)\n\nEncrypted environment variables have been removed for security reasons.\nSee https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions\n\nSetting environment variables from .travis.yml\n$ export NO_WATCH_TESTS=1\n$ export JOB_PART=integration\n\n$ export PATH=./node_modules/.bin:$PATH\nUpdating nvm\ntravis_fold:end:install.yarn [...]\nSetting up build cache [...]\n [...]\n$ node --version\nv6.13.0\n$ npm --version\n3.10.10\n$ nvm --version\n0.33.8\n$ yarn --version\n1.3.2\n$ npm run travis:$JOB_PART\n\n> webpack@4.0.1 travis:integration /home/travis/build/webpack/webpack\n> npm run cover:init && npm run cover:integration && npm run cover:report-min\n\n\n> webpack@4.0.1 cover:init /home/travis/build/webpack/webpack\n> rimraf coverage\n\n\n> webpack@4.0.1 cover:integration /home/travis/build/webpack/webpack\n> node --max-old-space-size=4096 --harmony --trace-deprecation ./node_modules/istanbul/lib/cli.js cover --report none node_modules/mocha/bin/_mocha -- test/*.test.js\n\n\n․․․․․․․․․(node:4135) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead\n    at UglifyJsPlugin.apply (/home/travis/build/webpack/webpack/node_modules/uglifyjs-webpack-plugin/dist/index.js:106:16)\n    at Object.apply (/home/travis/build/webpack/webpack/lib/WebpackOptionsDefaulter.js:9:14705)\n    at WebpackOptionsApply.process (/home/travis/build/webpack/webpack/lib/WebpackOptionsApply.js:9:24037)\n    at webpack (/home/travis/build/webpack/webpack/lib/webpack.js:9:2867)\n    at Context.<anonymous> (/home/travis/build/webpack/webpack/test/Compiler.test.js:250:20)\n    at callFnAsync (/home/travis/build/webpack/webpack/node_modules/mocha/lib/runnable.js:371:21)\n    at Test.Runnable.run (/home/travis/build/webpack/webpack/node_modules/mocha/lib/runnable.js:318:7)\n    at Runner.runTest (/home/travis/build/webpack/webpack/node_modules/mocha/lib/runner.js:443:10)\n    at /home/travis/build/webpack/webpack/node_modules/mocha/lib/runner.js:549:12\n    at /home/travis/build/webpack/webpack/node_modules/mocha/lib/runner.js:371:7\n    at Immediate.<anonymous> (/home/travis/build/webpack/webpack/node_modules/mocha/lib/runner.js:339:5)\n    at tryOnImmediate (timers.js:645:5)\n    at processImmediate [as _immediateCallback] (timers.js:617:5)\n․․․․․,,․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․Unable to CPU profile in < node 8.0\n․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․,․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․(node:4135) DeprecationWarning: Tapable.apply is deprecated. Call apply on the plugin directly instead\n    at Object.pitch (/home/travis/build/webpack/webpack/node_modules/worker-loader/dist/index.js:87:19)\n    at LOADER_EXECUTION (/home/travis/build/webpack/webpack/node_modules/loader-runner/lib/LoaderRunner.js:119:14)\n    at runSyncOrAsync (/home/travis/build/webpack/webpack/node_modules/loader-runner/lib/LoaderRunner.js:120:4)\n    at /home/travis/build/webpack/webpack/node_modules/loader-runner/lib/LoaderRunner.js:175:3\n    at loadLoader (/home/travis/build/webpack/webpack/node_modules/loader-runner/lib/loadLoader.js:36:3)\n    at iteratePitchingLoaders (/home/travis/build/webpack/webpack/node_modules/loader-runner/lib/LoaderRunner.js:169:2)\n    at runLoaders (/home/travis/build/webpack/webpack/node_modules/loader-runner/lib/LoaderRunner.js:362:2)\n    at NormalModule.doBuild (/home/travis/build/webpack/webpack/lib/NormalModule.js:9:10352)\n    at Compilation.buildModule (/home/travis/build/webpack/webpack/lib/Compilation.js:9:18256)\n    at factory.create (/home/travis/build/webpack/webpack/lib/Compilation.js:9:29554)\n    at factory (/home/travis/build/webpack/webpack/lib/NormalModuleFactory.js:9:18945)\n    at AsyncSeriesWaterfallHook.eval [as callAsync] (<anonymous>:6:1)\n    at resolver (/home/travis/build/webpack/webpack/lib/NormalModuleFactory.js:9:7405)\n    at process.nextTick (/home/travis/build/webpack/webpack/lib/NormalModuleFactory.js:9:15996)\n    at _combinedTickCallback (internal/process/next_tick.js:73:7)\n    at process._tickCallback (internal/process/next_tick.js:104:9)\n\n․,․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․Segmentation fault (core dumped)\n\n\n\n\n\n\nThe command \"npm run travis:$JOB_PART\" exited with 1.\nstore build cache [...]\nDone. Your build exited with 1.\n```\n\nSee [complete report here](https://travis-ci.org/webpack/webpack/builds/348045696?utm_source&#x3D;github_status&amp;utm_medium&#x3D;notification)."}}
{"actor":{"display_login":"webpack-bot"}}