{"action":"created","comment":{"url":"https://api.github.com/repos/apache/kafka/pulls/comments/91596966","pull_request_review_id":12098097,"id":91596966,"diff_hunk":"@@ -98,4 +116,47 @@ public String toString() {\n         }\n         return sb.toString();\n     }\n+\n+    protected class NodeMetrics  {\n+        final StreamsMetrics metrics;\n+        final String metricGrpName;\n+        final Map<String, String> metricTags;\n+\n+        final Sensor nodeProcessTimeSensor;\n+        final Sensor nodePunctuateTimeSensor;\n+        final Sensor contextForwardSensor;\n+        final Sensor nodeTaskCreationSensor;\n+        final Sensor nodeTaskDestructionSensor;\n+\n+\n+        public NodeMetrics(StreamsMetrics metrics, String name, String sensorNamePrefix) {\n+            String scope = \"node\";\n+            this.metrics = metrics;\n+            this.metricGrpName = \"node-metrics-\" + name;\n+            this.metricTags = new LinkedHashMap<>();\n+            this.metricTags.put(\"node-id\", \"-\" + name);\n+\n+            // these are all latency metrics\n+            this.nodeProcessTimeSensor = metrics.addLatencySensor(scope, name, \"process\", Sensor.RecordLevel.SENSOR_DEBUG);\n+            this.nodePunctuateTimeSensor = metrics.addLatencySensor(scope, name, \"punctuate\", Sensor.RecordLevel.SENSOR_DEBUG);\n+\n+            this.contextForwardSensor = metrics.sensor(sensorNamePrefix + \"node-forward-time\" + name, Sensor.RecordLevel.SENSOR_DEBUG);\n+            this.contextForwardSensor.add(new MetricName(sensorNamePrefix + \"node-forward-creation-rate\", metricGrpName, \"The average per-second number of newly created tasks\", metricTags), new Rate(new Count()));","path":"streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorNode.java","position":111,"original_position":111,"commit_id":"e13d2ba3ba6e57554dd31aa7f029e666be6ab627","original_commit_id":"e13d2ba3ba6e57554dd31aa7f029e666be6ab627","user":{"login":"guozhangwang","id":10729597,"avatar_url":"https://avatars.githubusercontent.com/u/10729597?v=3","gravatar_id":"","url":"https://api.github.com/users/guozhangwang","html_url":"https://github.com/guozhangwang","followers_url":"https://api.github.com/users/guozhangwang/followers","following_url":"https://api.github.com/users/guozhangwang/following{/other_user}","gists_url":"https://api.github.com/users/guozhangwang/gists{/gist_id}","starred_url":"https://api.github.com/users/guozhangwang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guozhangwang/subscriptions","organizations_url":"https://api.github.com/users/guozhangwang/orgs","repos_url":"https://api.github.com/users/guozhangwang/repos","events_url":"https://api.github.com/users/guozhangwang/events{/privacy}","received_events_url":"https://api.github.com/users/guozhangwang/received_events","type":"User","site_admin":false},"body":"The explanation does not sound right to me. I think it is \"The average per-second number of records processed by this processor\". Also maybe we can rename the variable name to `nodeThroughputSensor` to be more meaningful?","created_at":"2016-12-08T20:10:50Z","updated_at":"2016-12-08T20:31:45Z","html_url":"https://github.com/apache/kafka/pull/1446#discussion_r91596966","pull_request_url":"https://api.github.com/repos/apache/kafka/pulls/1446","_links":{"self":{"href":"https://api.github.com/repos/apache/kafka/pulls/comments/91596966"},"html":{"href":"https://github.com/apache/kafka/pull/1446#discussion_r91596966"},"pull_request":{"href":"https://api.github.com/repos/apache/kafka/pulls/1446"}}},"pull_request":{"url":"https://api.github.com/repos/apache/kafka/pulls/1446","id":71853084,"html_url":"https://github.com/apache/kafka/pull/1446","diff_url":"https://github.com/apache/kafka/pull/1446.diff","patch_url":"https://github.com/apache/kafka/pull/1446.patch","issue_url":"https://api.github.com/repos/apache/kafka/issues/1446","number":1446,"state":"open","locked":false,"title":"KAFKA-3715: add granular metrics per node","user":{"login":"aartigupta","id":3031383,"avatar_url":"https://avatars.githubusercontent.com/u/3031383?v=3","gravatar_id":"","url":"https://api.github.com/users/aartigupta","html_url":"https://github.com/aartigupta","followers_url":"https://api.github.com/users/aartigupta/followers","following_url":"https://api.github.com/users/aartigupta/following{/other_user}","gists_url":"https://api.github.com/users/aartigupta/gists{/gist_id}","starred_url":"https://api.github.com/users/aartigupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aartigupta/subscriptions","organizations_url":"https://api.github.com/users/aartigupta/orgs","repos_url":"https://api.github.com/users/aartigupta/repos","events_url":"https://api.github.com/users/aartigupta/events{/privacy}","received_events_url":"https://api.github.com/users/aartigupta/received_events","type":"Organization","site_admin":false},"body":"Kafka Streams: add granular metrics per node, also expose ability to …\n…register non latency metrics in StreamsMetrics\n\nfrom https://github.com/apache/kafka/pull/1362#issuecomment-218326690-------\nWe can consider adding metrics for process / punctuate / commit rate at the granularity of each processor node in addition to the global rate mentioned above. This is very helpful in debugging.\n\nWe can consider adding rate / total cumulated metrics for context.forward indicating how many records were forwarded downstream from this processor node as well. This is helpful in debugging.\n\nWe can consider adding metrics for each stream partition's timestamp.\nThis is helpful in debugging.\n## Besides the latency metrics, we can also add throughput latency in terms of source records consumed.\n\nMore discussions here https://issues.apache.org/jira/browse/KAFKA-3715\n","created_at":"2016-05-30T05:06:28Z","updated_at":"2016-12-08T20:31:45Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2d86ef2907904e3c8deb6a93b8dd1fff80d6d526","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/kafka/pulls/1446/commits","review_comments_url":"https://api.github.com/repos/apache/kafka/pulls/1446/comments","review_comment_url":"https://api.github.com/repos/apache/kafka/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/kafka/issues/1446/comments","statuses_url":"https://api.github.com/repos/apache/kafka/statuses/e13d2ba3ba6e57554dd31aa7f029e666be6ab627","head":{"label":"aartigupta:trunk","ref":"trunk","sha":"e13d2ba3ba6e57554dd31aa7f029e666be6ab627","user":{"login":"aartigupta","id":3031383,"avatar_url":"https://avatars.githubusercontent.com/u/3031383?v=3","gravatar_id":"","url":"https://api.github.com/users/aartigupta","html_url":"https://github.com/aartigupta","followers_url":"https://api.github.com/users/aartigupta/followers","following_url":"https://api.github.com/users/aartigupta/following{/other_user}","gists_url":"https://api.github.com/users/aartigupta/gists{/gist_id}","starred_url":"https://api.github.com/users/aartigupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aartigupta/subscriptions","organizations_url":"https://api.github.com/users/aartigupta/orgs","repos_url":"https://api.github.com/users/aartigupta/repos","events_url":"https://api.github.com/users/aartigupta/events{/privacy}","received_events_url":"https://api.github.com/users/aartigupta/received_events","type":"Organization","site_admin":false},"repo":{"id":59973805,"name":"kafka","full_name":"aartigupta/kafka","owner":{"login":"aartigupta","id":3031383,"avatar_url":"https://avatars.githubusercontent.com/u/3031383?v=3","gravatar_id":"","url":"https://api.github.com/users/aartigupta","html_url":"https://github.com/aartigupta","followers_url":"https://api.github.com/users/aartigupta/followers","following_url":"https://api.github.com/users/aartigupta/following{/other_user}","gists_url":"https://api.github.com/users/aartigupta/gists{/gist_id}","starred_url":"https://api.github.com/users/aartigupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aartigupta/subscriptions","organizations_url":"https://api.github.com/users/aartigupta/orgs","repos_url":"https://api.github.com/users/aartigupta/repos","events_url":"https://api.github.com/users/aartigupta/events{/privacy}","received_events_url":"https://api.github.com/users/aartigupta/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/aartigupta/kafka","description":"Mirror of Apache Kafka","fork":true,"url":"https://api.github.com/repos/aartigupta/kafka","forks_url":"https://api.github.com/repos/aartigupta/kafka/forks","keys_url":"https://api.github.com/repos/aartigupta/kafka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/aartigupta/kafka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/aartigupta/kafka/teams","hooks_url":"https://api.github.com/repos/aartigupta/kafka/hooks","issue_events_url":"https://api.github.com/repos/aartigupta/kafka/issues/events{/number}","events_url":"https://api.github.com/repos/aartigupta/kafka/events","assignees_url":"https://api.github.com/repos/aartigupta/kafka/assignees{/user}","branches_url":"https://api.github.com/repos/aartigupta/kafka/branches{/branch}","tags_url":"https://api.github.com/repos/aartigupta/kafka/tags","blobs_url":"https://api.github.com/repos/aartigupta/kafka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/aartigupta/kafka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/aartigupta/kafka/git/refs{/sha}","trees_url":"https://api.github.com/repos/aartigupta/kafka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/aartigupta/kafka/statuses/{sha}","languages_url":"https://api.github.com/repos/aartigupta/kafka/languages","stargazers_url":"https://api.github.com/repos/aartigupta/kafka/stargazers","contributors_url":"https://api.github.com/repos/aartigupta/kafka/contributors","subscribers_url":"https://api.github.com/repos/aartigupta/kafka/subscribers","subscription_url":"https://api.github.com/repos/aartigupta/kafka/subscription","commits_url":"https://api.github.com/repos/aartigupta/kafka/commits{/sha}","git_commits_url":"https://api.github.com/repos/aartigupta/kafka/git/commits{/sha}","comments_url":"https://api.github.com/repos/aartigupta/kafka/comments{/number}","issue_comment_url":"https://api.github.com/repos/aartigupta/kafka/issues/comments{/number}","contents_url":"https://api.github.com/repos/aartigupta/kafka/contents/{+path}","compare_url":"https://api.github.com/repos/aartigupta/kafka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/aartigupta/kafka/merges","archive_url":"https://api.github.com/repos/aartigupta/kafka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/aartigupta/kafka/downloads","issues_url":"https://api.github.com/repos/aartigupta/kafka/issues{/number}","pulls_url":"https://api.github.com/repos/aartigupta/kafka/pulls{/number}","milestones_url":"https://api.github.com/repos/aartigupta/kafka/milestones{/number}","notifications_url":"https://api.github.com/repos/aartigupta/kafka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/aartigupta/kafka/labels{/name}","releases_url":"https://api.github.com/repos/aartigupta/kafka/releases{/id}","deployments_url":"https://api.github.com/repos/aartigupta/kafka/deployments","created_at":"2016-05-30T02:08:08Z","updated_at":"2016-11-30T18:17:12Z","pushed_at":"2016-12-07T11:32:20Z","git_url":"git://github.com/aartigupta/kafka.git","ssh_url":"git@github.com:aartigupta/kafka.git","clone_url":"https://github.com/aartigupta/kafka.git","svn_url":"https://github.com/aartigupta/kafka","homepage":null,"size":36628,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"trunk"}},"base":{"label":"apache:trunk","ref":"trunk","sha":"9e72c12e9811811997b6ca99a6dc56a9af46e43d","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":2211243,"name":"kafka","full_name":"apache/kafka","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/kafka","description":"Mirror of Apache Kafka","fork":false,"url":"https://api.github.com/repos/apache/kafka","forks_url":"https://api.github.com/repos/apache/kafka/forks","keys_url":"https://api.github.com/repos/apache/kafka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/kafka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/kafka/teams","hooks_url":"https://api.github.com/repos/apache/kafka/hooks","issue_events_url":"https://api.github.com/repos/apache/kafka/issues/events{/number}","events_url":"https://api.github.com/repos/apache/kafka/events","assignees_url":"https://api.github.com/repos/apache/kafka/assignees{/user}","branches_url":"https://api.github.com/repos/apache/kafka/branches{/branch}","tags_url":"https://api.github.com/repos/apache/kafka/tags","blobs_url":"https://api.github.com/repos/apache/kafka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/kafka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/kafka/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/kafka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/kafka/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/kafka/languages","stargazers_url":"https://api.github.com/repos/apache/kafka/stargazers","contributors_url":"https://api.github.com/repos/apache/kafka/contributors","subscribers_url":"https://api.github.com/repos/apache/kafka/subscribers","subscription_url":"https://api.github.com/repos/apache/kafka/subscription","commits_url":"https://api.github.com/repos/apache/kafka/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/kafka/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/kafka/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/kafka/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/kafka/contents/{+path}","compare_url":"https://api.github.com/repos/apache/kafka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/kafka/merges","archive_url":"https://api.github.com/repos/apache/kafka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/kafka/downloads","issues_url":"https://api.github.com/repos/apache/kafka/issues{/number}","pulls_url":"https://api.github.com/repos/apache/kafka/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/kafka/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/kafka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/kafka/labels{/name}","releases_url":"https://api.github.com/repos/apache/kafka/releases{/id}","deployments_url":"https://api.github.com/repos/apache/kafka/deployments","created_at":"2011-08-15T18:06:16Z","updated_at":"2016-12-08T15:27:53Z","pushed_at":"2016-12-08T20:23:22Z","git_url":"git://github.com/apache/kafka.git","ssh_url":"git@github.com:apache/kafka.git","clone_url":"https://github.com/apache/kafka.git","svn_url":"https://github.com/apache/kafka","homepage":null,"size":55504,"stargazers_count":4240,"watchers_count":4240,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2640,"mirror_url":"git://git.apache.org/kafka.git","open_issues_count":309,"forks":2640,"open_issues":309,"watchers":4240,"default_branch":"trunk"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/kafka/pulls/1446"},"html":{"href":"https://github.com/apache/kafka/pull/1446"},"issue":{"href":"https://api.github.com/repos/apache/kafka/issues/1446"},"comments":{"href":"https://api.github.com/repos/apache/kafka/issues/1446/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/kafka/pulls/1446/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/kafka/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/kafka/pulls/1446/commits"},"statuses":{"href":"https://api.github.com/repos/apache/kafka/statuses/e13d2ba3ba6e57554dd31aa7f029e666be6ab627"}}}}
{"actor":{"display_login":"guozhangwang"}}