{"action":"created","comment":{"url":"https://api.github.com/repos/nodejs/node/pulls/comments/69523857","id":69523857,"diff_hunk":"@@ -1,147 +1,280 @@\n-# Node.js core benchmark tests\n+# Node.js core benchmark\n \n-This folder contains benchmark tests to measure the performance for certain\n-Node.js APIs.\n+This folder contains benchmarks to measure the performance of the Node.js APIs.\n+\n+## Table of Content\n+\n+* [Prerequisites](#prerequisites)\n+* [Running benchmarks](#running-benchmarks)\n+ * [Running individual benchmarks](#running-individual-benchmarks)\n+ * [Running all benchmarks](#running-all-benchmarks)\n+ * [Comparing node versions](#comparing-node-versions)\n+ * [Comparing parameters](#comparing-parameters)\n+* [Creating a benchmark](#creating-a-benchmark)\n \n ## Prerequisites\n \n-Most of the http benchmarks require [`wrk`][wrk] and [`ab`][ab] (ApacheBench) being installed.\n-These may be available through your preferred package manager.\n+Most of the http benchmarks require [`wrk`][wrk] to be installed. It may be\n+available through your preferred package manager. If not `wrk` can be built\n+[from source][wrk] via `make`.\n+\n+The R scripts use `ggplot2` and `plyr`, you can install them using the\n+R REPL.\n \n-If they are not available:\n-- `wrk` may easily be built [from source][wrk] via `make`.\n-- `ab` is sometimes bundled in a package called `apache2-utils`.\n+```\n+$ R\n+$ install.packages(\"ggplot2\")\n+$ install.packages(\"plyr\")\n+```\n \n [wrk]: https://github.com/wg/wrk\n-[ab]: http://httpd.apache.org/docs/2.2/programs/ab.html\n \n-## How to run tests\n+## Running benchmarks\n+\n+### Running individual benchmarks\n \n-There are three ways to run benchmark tests:\n+This can be useful for debugging a benchmark or doing a quick performance\n+measure. But it does not provide the statistical information to make any\n+conclusions about the performance.\n \n-### Run all tests of a given type\n+Individual benchmarks can be executed by simply executing the benchmark script\n+with node.\n \n-For example, buffers:\n+```\n+$ node benchmark/buffers/buffer-tostring.js\n \n-```bash\n-node benchmark/common.js buffers\n+buffers/buffer-tostring.js n=10000000 len=0 arg=true: 62710590.393305704\n+buffers/buffer-tostring.js n=10000000 len=1 arg=true: 9178624.591787899\n+buffers/buffer-tostring.js n=10000000 len=64 arg=true: 7658962.8891432695\n+buffers/buffer-tostring.js n=10000000 len=1024 arg=true: 4136904.4060201733\n+buffers/buffer-tostring.js n=10000000 len=0 arg=false: 22974354.231509723\n+buffers/buffer-tostring.js n=10000000 len=1 arg=false: 11485945.656765845\n+buffers/buffer-tostring.js n=10000000 len=64 arg=false: 8718280.70650129\n+buffers/buffer-tostring.js n=10000000 len=1024 arg=false: 4103857.0726124765\n ```\n \n-The above command will find all scripts under `buffers` directory and require\n-each of them as a module. When a test script is required, it creates an instance\n-of `Benchmark` (a class defined in common.js). In the next tick, the `Benchmark`\n-constructor iterates through the configuration object property values and runs\n-the test function with each of the combined arguments in spawned processes. For\n-example, buffers/buffer-read.js has the following configuration:\n+Each line represents a single benchmark with parameters specified as\n+`${variable}=${value}`. Each configuration combination is executed in a separate\n+process. This ensures that benchmark results aren't affected by the execution\n+order due to v8 optimizations. **The last number is the rate of operations\n+(higher is better).**\n+\n+Furthermore you can specify a subset of the configurations, by setting them in\n+the process arguments:\n \n-```js\n-var bench = common.createBenchmark(main, {\n-    noAssert: [false, true],\n-    buffer: ['fast', 'slow'],\n-    type: ['UInt8', 'UInt16LE', 'UInt16BE',\n-        'UInt32LE', 'UInt32BE',\n-        'Int8', 'Int16LE', 'Int16BE',\n-        'Int32LE', 'Int32BE',\n-        'FloatLE', 'FloatBE',\n-        'DoubleLE', 'DoubleBE'],\n-        millions: [1]\n-});\n ```\n-The runner takes one item from each of the property array value to build a list\n-of arguments to run the main function. The main function will receive the conf\n-object as follows:\n-\n-- first run:\n-```js\n-    {   noAssert: false,\n-        buffer: 'fast',\n-        type: 'UInt8',\n-        millions: 1\n-    }\n-```\n-- second run:\n-```js\n-    {\n-        noAssert: false,\n-        buffer: 'fast',\n-        type: 'UInt16LE',\n-        millions: 1\n-    }\n+$ node benchmark/buffers/buffer-tostring.js len=1024\n+\n+buffers/buffer-tostring.js n=10000000 len=1024 arg=true: 3498295.68561504\n+buffers/buffer-tostring.js n=10000000 len=1024 arg=false: 3783071.1678948295\n ```\n-...\n \n-In this case, the main function will run 2*2*14*1 = 56 times. The console output\n-looks like the following:\n+### Running all benchmarks\n+\n+Similar to running individual benchmarks, a group of benchmarks can be executed\n+by using the `run.js` tool. Again this does not provide the statistical\n+information to make any conclusions.\n \n ```\n-buffers//buffer-read.js\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt8 millions=1: 271.83\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt16LE millions=1: 239.43\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt16BE millions=1: 244.57\n+$ node benchmark/run.js arrays\n+\n+arrays/var-int.js\n+arrays/var-int.js n=25 type=Array: 71.90148040747789\n+arrays/var-int.js n=25 type=Buffer: 92.89648382795582\n+...\n+\n+arrays/zero-float.js\n+arrays/zero-float.js n=25 type=Array: 75.46208316171496\n+arrays/zero-float.js n=25 type=Buffer: 101.62785630273159\n ...\n+\n+arrays/zero-int.js\n+arrays/zero-int.js n=25 type=Array: 72.31023859816062\n+arrays/zero-int.js n=25 type=Buffer: 90.49906662339653\n+...\n+```\n+\n+It is possible to execute more groups by adding extra process arguments.\n+```\n+$ node benchmark/run.js arrays buffers\n ```\n \n-The last number is the rate of operations. Higher is better.\n+### Comparing node versions\n+\n+To compare the effect of a new node version use the `compare.js` tool. This\n+will run each benchmark multiple times, making it possible to calculate\n+statistics on the performance measures.\n+\n+As an example on how to check for a possible performance improvement, the\n+[#5134](https://github.com/nodejs/node/pull/5134) pull request will be used as\n+an example. This pull request _claims_ to improve the performance of the\n+`string_decoder` module.\n+\n+First build two versions of node, one from the master branch (here called\n+`./node-master`) and another with the pull request applied (here called\n+`./node-pr-5135`).\n+\n+The `compare.js` tool will then produce a csv file with the benchmark results.\n \n-### Run an individual test\n+```\n+$ node benchmark/compare.js string_decoder --old ./node-master --new ./node-pr-5134 > compare-pr-5134.csv\n+```\n+\n+For analysing the benchmark results use the `compare.R` tool.\n \n-For example, buffer-slice.js:\n+```\n+$ cat compare-pr-5134.csv | Rscript benchmark/compare.R\n \n-```bash\n-node benchmark/buffers/buffer-read.js\n+                                                                                      improvement significant      p.value\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=1024 encoding=ascii           12.46 %         *** 1.165345e-04\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=1024 encoding=base64-ascii    24.70 %         *** 1.820615e-15\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=1024 encoding=base64-utf8     23.60 %         *** 2.105625e-12\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=1024 encoding=utf8            14.04 %         *** 1.291105e-07\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=128  encoding=ascii            6.70 %           * 2.928003e-02\n+...\n ```\n-The output:\n+\n+In the output _improvement_ is the relative improvement of the new version,\n+hopefully this is positive. _significant_ tells if there are enough\n+statistical evidence to validate the _improvement_. If there are enough evidence\n+then there will be at least one star (`*`), more stars is just better. **However\n+if there are no stars, then you shouldn't make any conclusions based on the\n+_improvement_.** Sometimes this is fine, for example if you are expecting there\n+to be no improvements, then there shouldn't be any stars.\n+\n+_For the statistically minded the R scripts makes an [independent/unpaired\n+2-group t-test][t-test], with the null hypothesis that the performance is the\n+same for both versions. The significant field will show a star if the p-value\n+is less than `0.05`._\n+\n+[t-test]: https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes.2C_unequal_variances\n+\n+The `compare.R` tool can also produce a box plot by using the `--plot filename`\n+option. In this case there are 48 different benchmark combinations, thus you\n+may want to filter the csv file. This can be done while benchmarking using the\n+`--set` parameter (e.g. `--set encoding=ascii`) or by filtering results\n+afterwards using tools such as `sed` or `grep`. In the `sed` case be sure to\n+keep the first line since that contains the header information.\n+\n ```\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt8 millions=1: 246.79\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt16LE millions=1: 240.11\n-buffers/buffer-read.js noAssert=false buffer=fast type=UInt16BE millions=1: 245.91\n+$ cat compare-pr-5134.csv | sed '1p;/encoding=ascii/!d' | Rscript benchmark/compare.R --plot compare-plot.png\n+\n+                                                                               improvement significant      p.value\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=1024 encoding=ascii    12.46 %         *** 1.165345e-04\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=128 encoding=ascii      6.70 %           * 2.928003e-02\n+string_decoder/string-decoder.js n=250000 chunk=1024 inlen=32 encoding=ascii       7.47 %         *** 5.780583e-04\n+string_decoder/string-decoder.js n=250000 chunk=16 inlen=1024 encoding=ascii       8.94 %         *** 1.788579e-04\n+string_decoder/string-decoder.js n=250000 chunk=16 inlen=128 encoding=ascii       10.54 %         *** 4.016172e-05\n ...\n ```\n \n-### Run tests with options\n+![compare tool boxplot](doc_img/compare-boxplot.png)\n+\n+### Comparing parameters\n+\n+It can be useful to compare the performance for different parameters, for\n+example to analyze the time complexity.\n+\n+To do this use the `scatter.js` tool, this will run a benchmark multiple times\n+and generate a csv with the results.\n+\n+```\n+$ node benchmark/scatter.js benchmark/string_decoder/string-decoder.js > scatter.csv\n+```\n+\n+After generating the csv, a comparison table can be created using the\n+`scatter.R` tool. Even more useful it creates an actual scatter plot when using\n+the `--plot filename` option.\n \n-This example will run only the first type of url test, with one iteration.\n-(Note: benchmarks require __many__ iterations to be statistically accurate.)\n+```\n+$ cat scatter.csv | Rscript benchmark/scatter.R --xaxis chunk --category encoding --plot scatter-plot.png --log\n \n+aggregating variable: inlen\n \n-```bash\n-node benchmark/url/url-parse.js type=one n=1\n+chunk     encoding      mean confidence.interval\n+   16        ascii 1111933.3           221502.48\n+   16 base64-ascii  167508.4            33116.09\n+   16  base64-utf8  122666.6            25037.65\n+   16         utf8  783254.8           159601.79\n+   64        ascii 2623462.9           399791.36\n+   64 base64-ascii  462008.3            85369.45\n+   64  base64-utf8  420108.4            85612.05\n+   64         utf8 1358327.5           235152.03\n+  256        ascii 3730343.4           371530.47\n+  256 base64-ascii  663281.2            80302.73\n+  256  base64-utf8  632911.7            81393.07\n+  256         utf8 1554216.9           236066.53\n+ 1024        ascii 4399282.0           186436.46\n+ 1024 base64-ascii  730426.6            63806.12\n+ 1024  base64-utf8  680954.3            68076.33\n+ 1024         utf8 1554832.5           237532.07\n ```\n-Output:\n+\n+Because the scatter plot can only show two variables (in this case _chunk_ and\n+_encoding_) the rest is aggregated. Sometimes aggregating is a problem, this\n+can be solved by filtering. This can be done while benchmarking using the\n+`--set` parameter (e.g. `--set encoding=ascii`) or by filtering results\n+afterwards using tools such as `sed` or `grep`. In the `sed` case be\n+sure to keep the first line since that contains the header information.\n+\n ```\n-url/url-parse.js type=one n=1: 1663.74402\n+$ cat scatter.csv | sed -E '1p;/([^,]+, ){3}128,/!d' | Rscript benchmark/scatter.R --xaxis chunk --category encoding --plot scatter-plot.png --log\n+\n+chunk     encoding       mean confidence.interval\n+   16        ascii  701285.96           21233.982\n+   16 base64-ascii  107719.07            3339.439\n+   16  base64-utf8   72966.95            2438.448\n+   16         utf8  475340.84           17685.450\n+   64        ascii 2554105.08           87067.132\n+   64 base64-ascii  330120.32            8551.707\n+   64  base64-utf8  249693.19            8990.493\n+   64         utf8 1128671.90           48433.862\n+  256        ascii 4841070.04          181620.768\n+  256 base64-ascii  849545.53           29931.656\n+  256  base64-utf8  809629.89           33773.496\n+  256         utf8 1489525.15           49616.334\n+ 1024        ascii 4931512.12          165402.805\n+ 1024 base64-ascii  863933.22           27766.982\n+ 1024  base64-utf8  827093.97           24376.522\n+ 1024         utf8 1487176.43           50128.721\n ```\n \n-## How to write a benchmark test\n+![compare tool boxplot](doc_img/scatter-plot.png)\n \n-The benchmark tests are grouped by types. Each type corresponds to a subdirectory,\n-such as `arrays`, `buffers`, or `fs`.\n+## Creating a benchmark\n \n-Let's add a benchmark test for Buffer.slice function. We first create a file\n-buffers/buffer-slice.js.\n+All benchmarks use the `require('../common.js')` module. This contains the\n+`createBenchmark(main, configs)` method which will setup your benchmark.\n \n-### The code snippet\n+The first argument `main` is the benchmark function, the second argument\n+specifies the benchmark parameters. `createBenchmark` will run all possible\n+combinations of these parameters, unless specified otherwise. Note that the\n+configuration values can only be strings or numbers.\n \n-```js\n-var common = require('../common.js'); // Load the test runner\n+`createBenchmark` also creates a `bench` object, which is used for timing\n+the runtime of the benchmark. Run `bench.start()` after the initialization\n+and `bench.end(n)` when the benchmark is done. `n` is the number of operations\n+you performed in the benchmark.\n \n-var SlowBuffer = require('buffer').SlowBuffer;\n+```javascript","path":"benchmark/README.md","position":344,"original_position":344,"commit_id":"d10fbee4dd25c515aea0d6abba44dbb1e9af2860","original_commit_id":"d10fbee4dd25c515aea0d6abba44dbb1e9af2860","user":{"login":"Fishrock123","id":1093990,"avatar_url":"https://avatars.githubusercontent.com/u/1093990?v=3","gravatar_id":"","url":"https://api.github.com/users/Fishrock123","html_url":"https://github.com/Fishrock123","followers_url":"https://api.github.com/users/Fishrock123/followers","following_url":"https://api.github.com/users/Fishrock123/following{/other_user}","gists_url":"https://api.github.com/users/Fishrock123/gists{/gist_id}","starred_url":"https://api.github.com/users/Fishrock123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Fishrock123/subscriptions","organizations_url":"https://api.github.com/users/Fishrock123/orgs","repos_url":"https://api.github.com/users/Fishrock123/repos","events_url":"https://api.github.com/users/Fishrock123/events{/privacy}","received_events_url":"https://api.github.com/users/Fishrock123/received_events","type":"User","site_admin":false},"body":"can be ```js","created_at":"2016-07-05T08:28:18Z","updated_at":"2016-07-05T08:28:18Z","html_url":"https://github.com/nodejs/node/pull/7094#discussion_r69523857","pull_request_url":"https://api.github.com/repos/nodejs/node/pulls/7094","_links":{"self":{"href":"https://api.github.com/repos/nodejs/node/pulls/comments/69523857"},"html":{"href":"https://github.com/nodejs/node/pull/7094#discussion_r69523857"},"pull_request":{"href":"https://api.github.com/repos/nodejs/node/pulls/7094"}}},"pull_request":{"url":"https://api.github.com/repos/nodejs/node/pulls/7094","id":72199973,"html_url":"https://github.com/nodejs/node/pull/7094","diff_url":"https://github.com/nodejs/node/pull/7094.diff","patch_url":"https://github.com/nodejs/node/pull/7094.patch","issue_url":"https://api.github.com/repos/nodejs/node/issues/7094","number":7094,"state":"open","locked":false,"title":"Refactor benchmark tools for statistical significance","user":{"login":"AndreasMadsen","id":505333,"avatar_url":"https://avatars.githubusercontent.com/u/505333?v=3","gravatar_id":"","url":"https://api.github.com/users/AndreasMadsen","html_url":"https://github.com/AndreasMadsen","followers_url":"https://api.github.com/users/AndreasMadsen/followers","following_url":"https://api.github.com/users/AndreasMadsen/following{/other_user}","gists_url":"https://api.github.com/users/AndreasMadsen/gists{/gist_id}","starred_url":"https://api.github.com/users/AndreasMadsen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndreasMadsen/subscriptions","organizations_url":"https://api.github.com/users/AndreasMadsen/orgs","repos_url":"https://api.github.com/users/AndreasMadsen/repos","events_url":"https://api.github.com/users/AndreasMadsen/events{/privacy}","received_events_url":"https://api.github.com/users/AndreasMadsen/received_events","type":"User","site_admin":false},"body":"##### Checklist\r\n\r\n- [x] tests and code linting passes\r\n- [ ] a test and/or benchmark is included\r\n- [x] documentation is changed or added\r\n- [x] the commit message follows commit guidelines\r\n\r\n##### Affected core subsystem(s)\r\nbenchmark\r\n\r\n##### Description of change\r\n\r\nI have been rather confused about the benchmark suite and I don't think it is as user friendly as the rest of nodecore. This PR attempt to remove most of the confusion I was facing when I started using it. Primarily it:\r\n\r\n* removes unused/undocumented files\r\n* allows partially setting the benchmarks variables using process arguments.\r\n* refactor `compare.js` such comparing node versions and getting statistical significance is easy.\r\n* refactor the `plot.R` tool (now called `scatter`) to show a scatter plot with confidence bars.\r\n* refactor cli tools such the cli API is more homogeneous.\r\n* documents all the tools.\r\n* removes the implicit `process.exit(0)` after `bench.end()`.\r\n* uses `process.send` to avoid most parsing (the benchmark cli arguments haven't changed).\r\n\r\nThe specifics are documented in the commit messages. Please also see the the [new README](https://github.com/AndreasMadsen/node/blob/benchmark-refactor/benchmark/README.md) as quite a lot have changed (be sure the to check my spelling!).\r\n\r\n_Note that some benchmark takes a very long time to complete, e.g. `timers/timers.js\ttype=depth thousands=500` takes 11.25 min. Thus running it 30 times for statistical significance is unreasonable. I suspect the only reason why it is set to so many iterations is to get a small variance, but with the the new `compare` tool the variance can be estimated instead of being reduced. Thus we can reduce the number of iterations and still get the information we need. But I suggest we do that in another pull request, as is very different discussion._\r\n\r\n**Motivation (long story)**: I wanted to benchmark the effect of some `async_wrap` changes. I went to the `benchmark/` directory and read the [README](https://github.com/nodejs/node/blob/master/benchmark/README.md). However I quickly discovered that it was primarily about running benchmarks a single time and how to write benchmarks. And most importantly it didn't explain how to compare two node versions. _This is now documented in the [new README](https://github.com/AndreasMadsen/node/blob/benchmark-refactor/benchmark/README.md)._\r\n\r\nI then had to search for the tools myself and discovered the large amount of benchmarks files which where not put into categorized directories. I assumed they where somehow extra significant, but in reality they just appear to be unused. _These files are now removed._\r\n\r\nAfter discovering the compare tool, which has the cli API\r\n```\r\nnode benchmark/compare.js\r\n            <node-binary1> <node-binary2> +\r\n            [--html] [--red|-r] [--green|-g] +\r\n            [-- <type> [testFilter]]\r\n```\r\nI was confused about what the `--red`, `--green` was and how the `node-binary1` and `node-binary2` compared, should I write `./node-old ./node-new` or `./node-new ./node-old` if I wanted a positive improvement factor to signify an improvement? _The new compare API is_:\r\n```\r\nusage: ./node benchmark/compare.js <type> ...\r\n  --new    ./new-node-binary  new node binary (required)\r\n  --old    ./old-node-binary  old node binary (required)\r\n  --runs   30                 number of samples\r\n  --filter pattern            string to filter benchmark scripts\r\n  --var    variable=value     set benchmark variable (can be repeated)\r\n```\r\n\r\nAfter understanding `common.js` this it was still unclear if the performance was statistically significant different. I tried running the benchmark 5 times and got that 4/5 was an improvement, I was expecting it to have the same performance or be slower. (spoiler: it wasn't significant). _The compare.js script now runs the benchmarks many times (30 by default) and there is an R script to analyse the csv results._\r\n\r\nAt this point I wanted to do a rewrite of the benchmark tools (not the benchmarks themself) and changed a few other things in the process as well. - I'm a mathematician so I care a lot about statistical significance :)\r\n","created_at":"2016-06-01T15:18:38Z","updated_at":"2016-07-05T08:28:18Z","closed_at":null,"merged_at":null,"merge_commit_sha":"67300510f91c1882248338bd5e313dc72cca59f5","assignee":{"login":"Fishrock123","id":1093990,"avatar_url":"https://avatars.githubusercontent.com/u/1093990?v=3","gravatar_id":"","url":"https://api.github.com/users/Fishrock123","html_url":"https://github.com/Fishrock123","followers_url":"https://api.github.com/users/Fishrock123/followers","following_url":"https://api.github.com/users/Fishrock123/following{/other_user}","gists_url":"https://api.github.com/users/Fishrock123/gists{/gist_id}","starred_url":"https://api.github.com/users/Fishrock123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Fishrock123/subscriptions","organizations_url":"https://api.github.com/users/Fishrock123/orgs","repos_url":"https://api.github.com/users/Fishrock123/repos","events_url":"https://api.github.com/users/Fishrock123/events{/privacy}","received_events_url":"https://api.github.com/users/Fishrock123/received_events","type":"User","site_admin":false},"assignees":[{"login":"Fishrock123","id":1093990,"avatar_url":"https://avatars.githubusercontent.com/u/1093990?v=3","gravatar_id":"","url":"https://api.github.com/users/Fishrock123","html_url":"https://github.com/Fishrock123","followers_url":"https://api.github.com/users/Fishrock123/followers","following_url":"https://api.github.com/users/Fishrock123/following{/other_user}","gists_url":"https://api.github.com/users/Fishrock123/gists{/gist_id}","starred_url":"https://api.github.com/users/Fishrock123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Fishrock123/subscriptions","organizations_url":"https://api.github.com/users/Fishrock123/orgs","repos_url":"https://api.github.com/users/Fishrock123/repos","events_url":"https://api.github.com/users/Fishrock123/events{/privacy}","received_events_url":"https://api.github.com/users/Fishrock123/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/nodejs/node/pulls/7094/commits","review_comments_url":"https://api.github.com/repos/nodejs/node/pulls/7094/comments","review_comment_url":"https://api.github.com/repos/nodejs/node/pulls/comments{/number}","comments_url":"https://api.github.com/repos/nodejs/node/issues/7094/comments","statuses_url":"https://api.github.com/repos/nodejs/node/statuses/d10fbee4dd25c515aea0d6abba44dbb1e9af2860","head":{"label":"AndreasMadsen:benchmark-refactor","ref":"benchmark-refactor","sha":"d10fbee4dd25c515aea0d6abba44dbb1e9af2860","user":{"login":"AndreasMadsen","id":505333,"avatar_url":"https://avatars.githubusercontent.com/u/505333?v=3","gravatar_id":"","url":"https://api.github.com/users/AndreasMadsen","html_url":"https://github.com/AndreasMadsen","followers_url":"https://api.github.com/users/AndreasMadsen/followers","following_url":"https://api.github.com/users/AndreasMadsen/following{/other_user}","gists_url":"https://api.github.com/users/AndreasMadsen/gists{/gist_id}","starred_url":"https://api.github.com/users/AndreasMadsen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndreasMadsen/subscriptions","organizations_url":"https://api.github.com/users/AndreasMadsen/orgs","repos_url":"https://api.github.com/users/AndreasMadsen/repos","events_url":"https://api.github.com/users/AndreasMadsen/events{/privacy}","received_events_url":"https://api.github.com/users/AndreasMadsen/received_events","type":"User","site_admin":false},"repo":{"id":42876352,"name":"node","full_name":"AndreasMadsen/node","owner":{"login":"AndreasMadsen","id":505333,"avatar_url":"https://avatars.githubusercontent.com/u/505333?v=3","gravatar_id":"","url":"https://api.github.com/users/AndreasMadsen","html_url":"https://github.com/AndreasMadsen","followers_url":"https://api.github.com/users/AndreasMadsen/followers","following_url":"https://api.github.com/users/AndreasMadsen/following{/other_user}","gists_url":"https://api.github.com/users/AndreasMadsen/gists{/gist_id}","starred_url":"https://api.github.com/users/AndreasMadsen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndreasMadsen/subscriptions","organizations_url":"https://api.github.com/users/AndreasMadsen/orgs","repos_url":"https://api.github.com/users/AndreasMadsen/repos","events_url":"https://api.github.com/users/AndreasMadsen/events{/privacy}","received_events_url":"https://api.github.com/users/AndreasMadsen/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/AndreasMadsen/node","description":"Node.js JavaScript runtime :sparkles::turtle::rocket::sparkles:","fork":true,"url":"https://api.github.com/repos/AndreasMadsen/node","forks_url":"https://api.github.com/repos/AndreasMadsen/node/forks","keys_url":"https://api.github.com/repos/AndreasMadsen/node/keys{/key_id}","collaborators_url":"https://api.github.com/repos/AndreasMadsen/node/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/AndreasMadsen/node/teams","hooks_url":"https://api.github.com/repos/AndreasMadsen/node/hooks","issue_events_url":"https://api.github.com/repos/AndreasMadsen/node/issues/events{/number}","events_url":"https://api.github.com/repos/AndreasMadsen/node/events","assignees_url":"https://api.github.com/repos/AndreasMadsen/node/assignees{/user}","branches_url":"https://api.github.com/repos/AndreasMadsen/node/branches{/branch}","tags_url":"https://api.github.com/repos/AndreasMadsen/node/tags","blobs_url":"https://api.github.com/repos/AndreasMadsen/node/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/AndreasMadsen/node/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/AndreasMadsen/node/git/refs{/sha}","trees_url":"https://api.github.com/repos/AndreasMadsen/node/git/trees{/sha}","statuses_url":"https://api.github.com/repos/AndreasMadsen/node/statuses/{sha}","languages_url":"https://api.github.com/repos/AndreasMadsen/node/languages","stargazers_url":"https://api.github.com/repos/AndreasMadsen/node/stargazers","contributors_url":"https://api.github.com/repos/AndreasMadsen/node/contributors","subscribers_url":"https://api.github.com/repos/AndreasMadsen/node/subscribers","subscription_url":"https://api.github.com/repos/AndreasMadsen/node/subscription","commits_url":"https://api.github.com/repos/AndreasMadsen/node/commits{/sha}","git_commits_url":"https://api.github.com/repos/AndreasMadsen/node/git/commits{/sha}","comments_url":"https://api.github.com/repos/AndreasMadsen/node/comments{/number}","issue_comment_url":"https://api.github.com/repos/AndreasMadsen/node/issues/comments{/number}","contents_url":"https://api.github.com/repos/AndreasMadsen/node/contents/{+path}","compare_url":"https://api.github.com/repos/AndreasMadsen/node/compare/{base}...{head}","merges_url":"https://api.github.com/repos/AndreasMadsen/node/merges","archive_url":"https://api.github.com/repos/AndreasMadsen/node/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/AndreasMadsen/node/downloads","issues_url":"https://api.github.com/repos/AndreasMadsen/node/issues{/number}","pulls_url":"https://api.github.com/repos/AndreasMadsen/node/pulls{/number}","milestones_url":"https://api.github.com/repos/AndreasMadsen/node/milestones{/number}","notifications_url":"https://api.github.com/repos/AndreasMadsen/node/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/AndreasMadsen/node/labels{/name}","releases_url":"https://api.github.com/repos/AndreasMadsen/node/releases{/id}","deployments_url":"https://api.github.com/repos/AndreasMadsen/node/deployments","created_at":"2015-09-21T15:48:29Z","updated_at":"2016-03-18T20:12:28Z","pushed_at":"2016-06-29T13:05:46Z","git_url":"git://github.com/AndreasMadsen/node.git","ssh_url":"git@github.com:AndreasMadsen/node.git","clone_url":"https://github.com/AndreasMadsen/node.git","svn_url":"https://github.com/AndreasMadsen/node","homepage":"https://nodejs.org","size":211566,"stargazers_count":0,"watchers_count":0,"language":"JavaScript","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"nodejs:master","ref":"master","sha":"92dab4a2b24bf9d092485b09868ac3183c8998f8","user":{"login":"nodejs","id":9950313,"avatar_url":"https://avatars.githubusercontent.com/u/9950313?v=3","gravatar_id":"","url":"https://api.github.com/users/nodejs","html_url":"https://github.com/nodejs","followers_url":"https://api.github.com/users/nodejs/followers","following_url":"https://api.github.com/users/nodejs/following{/other_user}","gists_url":"https://api.github.com/users/nodejs/gists{/gist_id}","starred_url":"https://api.github.com/users/nodejs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nodejs/subscriptions","organizations_url":"https://api.github.com/users/nodejs/orgs","repos_url":"https://api.github.com/users/nodejs/repos","events_url":"https://api.github.com/users/nodejs/events{/privacy}","received_events_url":"https://api.github.com/users/nodejs/received_events","type":"Organization","site_admin":false},"repo":{"id":27193779,"name":"node","full_name":"nodejs/node","owner":{"login":"nodejs","id":9950313,"avatar_url":"https://avatars.githubusercontent.com/u/9950313?v=3","gravatar_id":"","url":"https://api.github.com/users/nodejs","html_url":"https://github.com/nodejs","followers_url":"https://api.github.com/users/nodejs/followers","following_url":"https://api.github.com/users/nodejs/following{/other_user}","gists_url":"https://api.github.com/users/nodejs/gists{/gist_id}","starred_url":"https://api.github.com/users/nodejs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nodejs/subscriptions","organizations_url":"https://api.github.com/users/nodejs/orgs","repos_url":"https://api.github.com/users/nodejs/repos","events_url":"https://api.github.com/users/nodejs/events{/privacy}","received_events_url":"https://api.github.com/users/nodejs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/nodejs/node","description":"Node.js JavaScript runtime :sparkles::turtle::rocket::sparkles:","fork":false,"url":"https://api.github.com/repos/nodejs/node","forks_url":"https://api.github.com/repos/nodejs/node/forks","keys_url":"https://api.github.com/repos/nodejs/node/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nodejs/node/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nodejs/node/teams","hooks_url":"https://api.github.com/repos/nodejs/node/hooks","issue_events_url":"https://api.github.com/repos/nodejs/node/issues/events{/number}","events_url":"https://api.github.com/repos/nodejs/node/events","assignees_url":"https://api.github.com/repos/nodejs/node/assignees{/user}","branches_url":"https://api.github.com/repos/nodejs/node/branches{/branch}","tags_url":"https://api.github.com/repos/nodejs/node/tags","blobs_url":"https://api.github.com/repos/nodejs/node/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nodejs/node/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nodejs/node/git/refs{/sha}","trees_url":"https://api.github.com/repos/nodejs/node/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nodejs/node/statuses/{sha}","languages_url":"https://api.github.com/repos/nodejs/node/languages","stargazers_url":"https://api.github.com/repos/nodejs/node/stargazers","contributors_url":"https://api.github.com/repos/nodejs/node/contributors","subscribers_url":"https://api.github.com/repos/nodejs/node/subscribers","subscription_url":"https://api.github.com/repos/nodejs/node/subscription","commits_url":"https://api.github.com/repos/nodejs/node/commits{/sha}","git_commits_url":"https://api.github.com/repos/nodejs/node/git/commits{/sha}","comments_url":"https://api.github.com/repos/nodejs/node/comments{/number}","issue_comment_url":"https://api.github.com/repos/nodejs/node/issues/comments{/number}","contents_url":"https://api.github.com/repos/nodejs/node/contents/{+path}","compare_url":"https://api.github.com/repos/nodejs/node/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nodejs/node/merges","archive_url":"https://api.github.com/repos/nodejs/node/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nodejs/node/downloads","issues_url":"https://api.github.com/repos/nodejs/node/issues{/number}","pulls_url":"https://api.github.com/repos/nodejs/node/pulls{/number}","milestones_url":"https://api.github.com/repos/nodejs/node/milestones{/number}","notifications_url":"https://api.github.com/repos/nodejs/node/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nodejs/node/labels{/name}","releases_url":"https://api.github.com/repos/nodejs/node/releases{/id}","deployments_url":"https://api.github.com/repos/nodejs/node/deployments","created_at":"2014-11-26T19:57:11Z","updated_at":"2016-07-05T07:09:23Z","pushed_at":"2016-07-05T05:35:38Z","git_url":"git://github.com/nodejs/node.git","ssh_url":"git@github.com:nodejs/node.git","clone_url":"https://github.com/nodejs/node.git","svn_url":"https://github.com/nodejs/node","homepage":"https://nodejs.org","size":289059,"stargazers_count":24585,"watchers_count":24585,"language":"JavaScript","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3651,"mirror_url":null,"open_issues_count":691,"forks":3651,"open_issues":691,"watchers":24585,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/nodejs/node/pulls/7094"},"html":{"href":"https://github.com/nodejs/node/pull/7094"},"issue":{"href":"https://api.github.com/repos/nodejs/node/issues/7094"},"comments":{"href":"https://api.github.com/repos/nodejs/node/issues/7094/comments"},"review_comments":{"href":"https://api.github.com/repos/nodejs/node/pulls/7094/comments"},"review_comment":{"href":"https://api.github.com/repos/nodejs/node/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/nodejs/node/pulls/7094/commits"},"statuses":{"href":"https://api.github.com/repos/nodejs/node/statuses/d10fbee4dd25c515aea0d6abba44dbb1e9af2860"}}}}
{"actor":{"display_login":"Fishrock123"}}