{"action":"created","comment":{"url":"https://api.github.com/repos/netty/netty/pulls/comments/197082192","pull_request_review_id":130752812,"id":197082192,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5NzA4MjE5Mg==","diff_hunk":"@@ -54,16 +45,19 @@ public void destroy() {\n \n     @Benchmark\n     public void codecOutList() {\n+        codecOutputList = CodecOutputList.newInstance();\n         benchmarkAddAndClear(codecOutputList, elements);","path":"microbench/src/main/java/io/netty/handler/codec/CodecOutputListBenchmark.java","position":30,"original_position":30,"commit_id":"1935cc3593e83ad9c1d012decf24406d4a7dce32","original_commit_id":"1935cc3593e83ad9c1d012decf24406d4a7dce32","user":{"login":"normanmaurer","id":439362,"node_id":"MDQ6VXNlcjQzOTM2Mg==","avatar_url":"https://avatars3.githubusercontent.com/u/439362?v=4","gravatar_id":"","url":"https://api.github.com/users/normanmaurer","html_url":"https://github.com/normanmaurer","followers_url":"https://api.github.com/users/normanmaurer/followers","following_url":"https://api.github.com/users/normanmaurer/following{/other_user}","gists_url":"https://api.github.com/users/normanmaurer/gists{/gist_id}","starred_url":"https://api.github.com/users/normanmaurer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/normanmaurer/subscriptions","organizations_url":"https://api.github.com/users/normanmaurer/orgs","repos_url":"https://api.github.com/users/normanmaurer/repos","events_url":"https://api.github.com/users/normanmaurer/events{/privacy}","received_events_url":"https://api.github.com/users/normanmaurer/received_events","type":"User","site_admin":false},"body":"true... lets just keep it as it is for now","created_at":"2018-06-21T10:18:31Z","updated_at":"2018-06-21T10:18:31Z","html_url":"https://github.com/netty/netty/pull/8005#discussion_r197082192","pull_request_url":"https://api.github.com/repos/netty/netty/pulls/8005","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/netty/netty/pulls/comments/197082192"},"html":{"href":"https://github.com/netty/netty/pull/8005#discussion_r197082192"},"pull_request":{"href":"https://api.github.com/repos/netty/netty/pulls/8005"}},"in_reply_to_id":195191983},"pull_request":{"url":"https://api.github.com/repos/netty/netty/pulls/8005","id":192640083,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkyNjQwMDgz","html_url":"https://github.com/netty/netty/pull/8005","diff_url":"https://github.com/netty/netty/pull/8005.diff","patch_url":"https://github.com/netty/netty/pull/8005.patch","issue_url":"https://api.github.com/repos/netty/netty/issues/8005","number":8005,"state":"open","locked":false,"title":"Avoiding the usage of Jmh Invocation level on setup/teardown ","user":{"login":"DiegoEliasCosta","id":466193,"node_id":"MDQ6VXNlcjQ2NjE5Mw==","avatar_url":"https://avatars0.githubusercontent.com/u/466193?v=4","gravatar_id":"","url":"https://api.github.com/users/DiegoEliasCosta","html_url":"https://github.com/DiegoEliasCosta","followers_url":"https://api.github.com/users/DiegoEliasCosta/followers","following_url":"https://api.github.com/users/DiegoEliasCosta/following{/other_user}","gists_url":"https://api.github.com/users/DiegoEliasCosta/gists{/gist_id}","starred_url":"https://api.github.com/users/DiegoEliasCosta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DiegoEliasCosta/subscriptions","organizations_url":"https://api.github.com/users/DiegoEliasCosta/orgs","repos_url":"https://api.github.com/users/DiegoEliasCosta/repos","events_url":"https://api.github.com/users/DiegoEliasCosta/events{/privacy}","received_events_url":"https://api.github.com/users/DiegoEliasCosta/received_events","type":"User","site_admin":false},"body":"Motivation:\r\n\r\nWe are conducting a scientific study to investigate bad practices/anti-patterns on creating micro-benchmarks using JMH, and we found two instances of harmfull usage of Invocation level in your benchmark classes.\r\n\r\nThe usage of Invocation level for JMH fixture methods (setup/teardown) inccurs in a significant impact in\r\nin the benchmark time (see [JMH Documentation ](http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-core/src/main/java/org/openjdk/jmh/annotations/Level.java)). \r\n\r\nWhen the benchmark and the setup/teardown is too small (less than a milisecond) the Invocation level might saturate the system with timestamp requests and iteration synchronizations which introduce artificial latency, throughput, and scalability bottlenecks.\r\n\r\nFrom our experiments Netty has two instances of such case, HeadersBenchmark and CodecInputListBenchmark. On those cases, the benchmark is far too small (<100ns) and the Invocation level setup offsets the measurement considerably.\r\n\r\nModification:\r\n\r\nI have applied the simplest fix patch to include what was previously in the setup, inside the benchmark code. \r\n\r\nResult:\r\n\r\nIn most cases, the benchmarks run faster now, without the artificial latency introduced by the setup code. I have attached a csv with all benchmarks runs and the difference between Netty benchmark on those patches \r\n- We run the entire benchmark 10 times with default parameters (iterations, forks, warmups kept as default)\r\n- Median score reported \r\n- Factor = 0 means no significant difference (Moore Median's Test)\r\n\r\nAttached Summary: [netty-invocation-10trials-summary.zip](https://github.com/netty/netty/files/2071662/netty-invocation-10trials-summary.zip)\r\n\r\nEnvironment: \r\nTests run on a Computational server with CPU: E5-1660-3.3GHZ  (6 cores + HT), 64 GB RAM.","created_at":"2018-06-05T09:16:29Z","updated_at":"2018-06-21T10:18:31Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a0530cb5fd9fdf01ca61016ac86172d88e17dca1","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/netty/netty/pulls/8005/commits","review_comments_url":"https://api.github.com/repos/netty/netty/pulls/8005/comments","review_comment_url":"https://api.github.com/repos/netty/netty/pulls/comments{/number}","comments_url":"https://api.github.com/repos/netty/netty/issues/8005/comments","statuses_url":"https://api.github.com/repos/netty/netty/statuses/1935cc3593e83ad9c1d012decf24406d4a7dce32","head":{"label":"DiegoEliasCosta:jmh-antipatterns-invocationlevel-fixtures","ref":"jmh-antipatterns-invocationlevel-fixtures","sha":"1935cc3593e83ad9c1d012decf24406d4a7dce32","user":{"login":"DiegoEliasCosta","id":466193,"node_id":"MDQ6VXNlcjQ2NjE5Mw==","avatar_url":"https://avatars0.githubusercontent.com/u/466193?v=4","gravatar_id":"","url":"https://api.github.com/users/DiegoEliasCosta","html_url":"https://github.com/DiegoEliasCosta","followers_url":"https://api.github.com/users/DiegoEliasCosta/followers","following_url":"https://api.github.com/users/DiegoEliasCosta/following{/other_user}","gists_url":"https://api.github.com/users/DiegoEliasCosta/gists{/gist_id}","starred_url":"https://api.github.com/users/DiegoEliasCosta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DiegoEliasCosta/subscriptions","organizations_url":"https://api.github.com/users/DiegoEliasCosta/orgs","repos_url":"https://api.github.com/users/DiegoEliasCosta/repos","events_url":"https://api.github.com/users/DiegoEliasCosta/events{/privacy}","received_events_url":"https://api.github.com/users/DiegoEliasCosta/received_events","type":"User","site_admin":false},"repo":{"id":131031755,"node_id":"MDEwOlJlcG9zaXRvcnkxMzEwMzE3NTU=","name":"netty","full_name":"DiegoEliasCosta/netty","owner":{"login":"DiegoEliasCosta","id":466193,"node_id":"MDQ6VXNlcjQ2NjE5Mw==","avatar_url":"https://avatars0.githubusercontent.com/u/466193?v=4","gravatar_id":"","url":"https://api.github.com/users/DiegoEliasCosta","html_url":"https://github.com/DiegoEliasCosta","followers_url":"https://api.github.com/users/DiegoEliasCosta/followers","following_url":"https://api.github.com/users/DiegoEliasCosta/following{/other_user}","gists_url":"https://api.github.com/users/DiegoEliasCosta/gists{/gist_id}","starred_url":"https://api.github.com/users/DiegoEliasCosta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DiegoEliasCosta/subscriptions","organizations_url":"https://api.github.com/users/DiegoEliasCosta/orgs","repos_url":"https://api.github.com/users/DiegoEliasCosta/repos","events_url":"https://api.github.com/users/DiegoEliasCosta/events{/privacy}","received_events_url":"https://api.github.com/users/DiegoEliasCosta/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/DiegoEliasCosta/netty","description":"Netty project - an event-driven asynchronous network application framework","fork":true,"url":"https://api.github.com/repos/DiegoEliasCosta/netty","forks_url":"https://api.github.com/repos/DiegoEliasCosta/netty/forks","keys_url":"https://api.github.com/repos/DiegoEliasCosta/netty/keys{/key_id}","collaborators_url":"https://api.github.com/repos/DiegoEliasCosta/netty/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/DiegoEliasCosta/netty/teams","hooks_url":"https://api.github.com/repos/DiegoEliasCosta/netty/hooks","issue_events_url":"https://api.github.com/repos/DiegoEliasCosta/netty/issues/events{/number}","events_url":"https://api.github.com/repos/DiegoEliasCosta/netty/events","assignees_url":"https://api.github.com/repos/DiegoEliasCosta/netty/assignees{/user}","branches_url":"https://api.github.com/repos/DiegoEliasCosta/netty/branches{/branch}","tags_url":"https://api.github.com/repos/DiegoEliasCosta/netty/tags","blobs_url":"https://api.github.com/repos/DiegoEliasCosta/netty/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/DiegoEliasCosta/netty/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/DiegoEliasCosta/netty/git/refs{/sha}","trees_url":"https://api.github.com/repos/DiegoEliasCosta/netty/git/trees{/sha}","statuses_url":"https://api.github.com/repos/DiegoEliasCosta/netty/statuses/{sha}","languages_url":"https://api.github.com/repos/DiegoEliasCosta/netty/languages","stargazers_url":"https://api.github.com/repos/DiegoEliasCosta/netty/stargazers","contributors_url":"https://api.github.com/repos/DiegoEliasCosta/netty/contributors","subscribers_url":"https://api.github.com/repos/DiegoEliasCosta/netty/subscribers","subscription_url":"https://api.github.com/repos/DiegoEliasCosta/netty/subscription","commits_url":"https://api.github.com/repos/DiegoEliasCosta/netty/commits{/sha}","git_commits_url":"https://api.github.com/repos/DiegoEliasCosta/netty/git/commits{/sha}","comments_url":"https://api.github.com/repos/DiegoEliasCosta/netty/comments{/number}","issue_comment_url":"https://api.github.com/repos/DiegoEliasCosta/netty/issues/comments{/number}","contents_url":"https://api.github.com/repos/DiegoEliasCosta/netty/contents/{+path}","compare_url":"https://api.github.com/repos/DiegoEliasCosta/netty/compare/{base}...{head}","merges_url":"https://api.github.com/repos/DiegoEliasCosta/netty/merges","archive_url":"https://api.github.com/repos/DiegoEliasCosta/netty/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/DiegoEliasCosta/netty/downloads","issues_url":"https://api.github.com/repos/DiegoEliasCosta/netty/issues{/number}","pulls_url":"https://api.github.com/repos/DiegoEliasCosta/netty/pulls{/number}","milestones_url":"https://api.github.com/repos/DiegoEliasCosta/netty/milestones{/number}","notifications_url":"https://api.github.com/repos/DiegoEliasCosta/netty/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/DiegoEliasCosta/netty/labels{/name}","releases_url":"https://api.github.com/repos/DiegoEliasCosta/netty/releases{/id}","deployments_url":"https://api.github.com/repos/DiegoEliasCosta/netty/deployments","created_at":"2018-04-25T16:05:44Z","updated_at":"2018-04-25T16:05:57Z","pushed_at":"2018-06-05T08:56:38Z","git_url":"git://github.com/DiegoEliasCosta/netty.git","ssh_url":"git@github.com:DiegoEliasCosta/netty.git","clone_url":"https://github.com/DiegoEliasCosta/netty.git","svn_url":"https://github.com/DiegoEliasCosta/netty","homepage":"http://netty.io/","size":50630,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"4.1"}},"base":{"label":"netty:4.1","ref":"4.1","sha":"b192bf12ad2af92bba0f32c9d3127b1192e54670","user":{"login":"netty","id":473791,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3Mzc5MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/473791?v=4","gravatar_id":"","url":"https://api.github.com/users/netty","html_url":"https://github.com/netty","followers_url":"https://api.github.com/users/netty/followers","following_url":"https://api.github.com/users/netty/following{/other_user}","gists_url":"https://api.github.com/users/netty/gists{/gist_id}","starred_url":"https://api.github.com/users/netty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/netty/subscriptions","organizations_url":"https://api.github.com/users/netty/orgs","repos_url":"https://api.github.com/users/netty/repos","events_url":"https://api.github.com/users/netty/events{/privacy}","received_events_url":"https://api.github.com/users/netty/received_events","type":"Organization","site_admin":false},"repo":{"id":1064563,"node_id":"MDEwOlJlcG9zaXRvcnkxMDY0NTYz","name":"netty","full_name":"netty/netty","owner":{"login":"netty","id":473791,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ3Mzc5MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/473791?v=4","gravatar_id":"","url":"https://api.github.com/users/netty","html_url":"https://github.com/netty","followers_url":"https://api.github.com/users/netty/followers","following_url":"https://api.github.com/users/netty/following{/other_user}","gists_url":"https://api.github.com/users/netty/gists{/gist_id}","starred_url":"https://api.github.com/users/netty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/netty/subscriptions","organizations_url":"https://api.github.com/users/netty/orgs","repos_url":"https://api.github.com/users/netty/repos","events_url":"https://api.github.com/users/netty/events{/privacy}","received_events_url":"https://api.github.com/users/netty/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/netty/netty","description":"Netty project - an event-driven asynchronous network application framework","fork":false,"url":"https://api.github.com/repos/netty/netty","forks_url":"https://api.github.com/repos/netty/netty/forks","keys_url":"https://api.github.com/repos/netty/netty/keys{/key_id}","collaborators_url":"https://api.github.com/repos/netty/netty/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/netty/netty/teams","hooks_url":"https://api.github.com/repos/netty/netty/hooks","issue_events_url":"https://api.github.com/repos/netty/netty/issues/events{/number}","events_url":"https://api.github.com/repos/netty/netty/events","assignees_url":"https://api.github.com/repos/netty/netty/assignees{/user}","branches_url":"https://api.github.com/repos/netty/netty/branches{/branch}","tags_url":"https://api.github.com/repos/netty/netty/tags","blobs_url":"https://api.github.com/repos/netty/netty/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/netty/netty/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/netty/netty/git/refs{/sha}","trees_url":"https://api.github.com/repos/netty/netty/git/trees{/sha}","statuses_url":"https://api.github.com/repos/netty/netty/statuses/{sha}","languages_url":"https://api.github.com/repos/netty/netty/languages","stargazers_url":"https://api.github.com/repos/netty/netty/stargazers","contributors_url":"https://api.github.com/repos/netty/netty/contributors","subscribers_url":"https://api.github.com/repos/netty/netty/subscribers","subscription_url":"https://api.github.com/repos/netty/netty/subscription","commits_url":"https://api.github.com/repos/netty/netty/commits{/sha}","git_commits_url":"https://api.github.com/repos/netty/netty/git/commits{/sha}","comments_url":"https://api.github.com/repos/netty/netty/comments{/number}","issue_comment_url":"https://api.github.com/repos/netty/netty/issues/comments{/number}","contents_url":"https://api.github.com/repos/netty/netty/contents/{+path}","compare_url":"https://api.github.com/repos/netty/netty/compare/{base}...{head}","merges_url":"https://api.github.com/repos/netty/netty/merges","archive_url":"https://api.github.com/repos/netty/netty/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/netty/netty/downloads","issues_url":"https://api.github.com/repos/netty/netty/issues{/number}","pulls_url":"https://api.github.com/repos/netty/netty/pulls{/number}","milestones_url":"https://api.github.com/repos/netty/netty/milestones{/number}","notifications_url":"https://api.github.com/repos/netty/netty/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/netty/netty/labels{/name}","releases_url":"https://api.github.com/repos/netty/netty/releases{/id}","deployments_url":"https://api.github.com/repos/netty/netty/deployments","created_at":"2010-11-09T09:22:21Z","updated_at":"2018-06-21T08:21:19Z","pushed_at":"2018-06-21T10:09:57Z","git_url":"git://github.com/netty/netty.git","ssh_url":"git@github.com:netty/netty.git","clone_url":"https://github.com/netty/netty.git","svn_url":"https://github.com/netty/netty","homepage":"http://netty.io/","size":51463,"stargazers_count":14505,"watchers_count":14505,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6466,"mirror_url":null,"archived":false,"open_issues_count":343,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":6466,"open_issues":343,"watchers":14505,"default_branch":"4.1"}},"_links":{"self":{"href":"https://api.github.com/repos/netty/netty/pulls/8005"},"html":{"href":"https://github.com/netty/netty/pull/8005"},"issue":{"href":"https://api.github.com/repos/netty/netty/issues/8005"},"comments":{"href":"https://api.github.com/repos/netty/netty/issues/8005/comments"},"review_comments":{"href":"https://api.github.com/repos/netty/netty/pulls/8005/comments"},"review_comment":{"href":"https://api.github.com/repos/netty/netty/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/netty/netty/pulls/8005/commits"},"statuses":{"href":"https://api.github.com/repos/netty/netty/statuses/1935cc3593e83ad9c1d012decf24406d4a7dce32"}},"author_association":"NONE"}}
{"actor":{"display_login":"normanmaurer"}}