{"action":"created","comment":{"url":"https://api.github.com/repos/apache/flink/pulls/comments/68564218","id":68564218,"diff_hunk":"@@ -1350,3 +1350,211 @@ You may specify program arguments before the job is executed. The plan visualiza\n the execution plan before executing the Flink job.\n \n {% top %}\n+\n+Metrics\n+-------------------\n+\n+Flink exposes a metric system that allows users gather and expose metrics to external systems.\n+\n+### Registering metrics\n+\n+You can access the metric system from any user function that extends [RichFunction]({{ site.baseurl }}/apis/common/index.html#rich-functions) by calling getRuntimeContext().getMetricGroup().\n+This method returns a MetricGroup object on which you can create and register new metrics.\n+\n+If you want to count the number of records your user function has received you could use a Counter like this:\n+\n+{% highlight java %}\n+\n+public class MyMapper extends RichMapFunction<String, Integer> {\n+  private Counter counter;\n+\n+  @Override\n+  public void open(Configuration config) {\n+    // create and register a counter\n+    this.counter = getRuntimeContext().getMetricGroup().counter(\"myCounter\");\n+    ...\n+  }\n+\n+  @public Integer map(String value) throws Exception {\n+    // increment counter\n+    this.counter.inc();\n+    ...\n+  }\n+}\n+\n+{% endhighlight %}\n+\n+### Metric types\n+\n+Flink supports Counters, Gauges and Histograms.\n+\n+A Counter is used to count something. The current value can be in- or decremented using `inc([long n])` or `dec([long n])`.\n+You can create and registers a Counter by calling `counter(String name)` on a MetricGroup. Alternatively, you can pass an instance of your own Counter implementation along with the name.\n+\n+A Gauge provides a value of any type on demand. In order to use a Gauge you must first create a class that implements the `org.apache.flink.metrics.Gauge` interface.\n+There is not restriction for the type of the returned value.\n+You can register a gauge by calling `gauge(String name, Gauge gauge)` on a MetricGroup.\n+\n+A Histogram measure the distribution of long values.\n+You can register one by calling histogram(String name, Histogram histogram) on a MetricGroup.\n+\n+Flink only provides an interface for Histograms, but offers a Wrapper that allows usage of Codahale/DropWizard Histograms. (org.apache.flink.dropwizard.metrics.DropWizardHistogramWrapper)\n+\n+### Scope\n+\n+Every registered metric has an automatically assigned scope which represent the entities it is tied to. By default a metric that is registered in a user function will be scoped to the Operator in which the function runs, the Task/Job it belongs to and the TaskManager/Host it is executed on. This is referred to as the \"system scope\".\n+\n+You can define an additonal \"user scope\" by calling the MetricGroup#addGroup((int/String) name) method.\n+\n+{% highlight java %}\n+\n+counter2 = getRuntimeContext().getMetricGroup().addGroup(\"MyMetrics\").counter(\"myCounter2\");\n+\n+{% endhighlight %}\n+\n+The name under which a metric is exported is based on both scopes and the name passed in the `counter() call. The order is always \\<system_scope>\\<user_scope>\\<name>.\n+\n+The system scope allows the reported name to contain contextual information like the name of job it was registered in without requiring the user to pass this information manually.\n+\n+How the system scope affects the reported name for a metric can be modified by setting the following keys in the flink-conf.yaml. \n+Each of these keys expect a format string that may contain constants (e.g. \"taskmanager\") and variables (e.g. \"\\<task_id>\") which will be replaced at runtime.\n+\n+- `metrics.scope.jm`\n+  - Default: \\<host>.jobmanager\n+- `metrics.scope.jm.job`\n+  - Default: \\<host>.jobmanager.\\<job_name>\n+- `metrics.scope.tm`\n+  - Default: \\<host>.taskmanager.\\<tm_id>\n+- `metrics.scope.tm.job`\n+  - Default: \\<host>.taskmanager.\\<tm_id>.\\<job_name>\n+- `metrics.scope.task`\n+  - Default: \\<host>.taskmanager.\\<tm_id>.\\<job_name>.\\<task_name>.\\<subtask_index>\n+- `metrics.scope.operator`\n+  - Default: \\<host>.taskmanager.\\<tm_id>.\\<job_name>.\\<operator_name>.\\<subtask_index>\n+\n+The following is a list of all variables available:\n+\n+- JobManager: \\<host>\n+- TaskManager: \\<host>, \\<tm_id>\n+- Job: \\<job_id>, \\<job_name>\n+- Task: \\<task_id>, \\<task_name>, \\<task_attempt_id>, \\<task_attempt_num>, \\<subtask_index>\n+- Operator: \\<operator_name>, \\<subtask_index>\n+\n+There is no restriction on the order of variables.\n+\n+Which format is applied to a metric depends on which entities it was scoped to.\n+The `metrics.scope.operator` format will be applied to all metrics that were scoped to an Operator.\n+The `metrics.scope.tm.job` format will be applied to all metrics that were only scoped to a Job and a TaskManager.\n+\n+This allows you to define explicitly how metrics on every level should be reported.\n+\n+### Reporter\n+\n+Metrics can be exposed to an external system by configuring a reporter in the `conf/flink-conf.yaml`.\n+\n+- `metrics.reporter.class`: The class of the reporter to use.\n+  - Example: org.apache.flink.metrics.reporter.JMXReporter)\n+- `metrics.reporter.arguments`: A list of named parameters that are passed to the reporter.\n+  - Example: --host localhost --port 9010)\n+- `metrics.reporter.interval`: The interval between reports.\n+  - Example: 10 SECONDS)\n+\n+By default Flink uses JMX to expose metrics.\n+\n+The port for JMX can be configured by setting the `metrics.jmx.port` key. This parameter expects either a single port\n+or a port range, with the default being 9010-9025. Which port is used in the end is shown in the relevant Job-/TaskManager log.\n+\n+All non-JMXReporters are not part of the distribution and have to be added to the classpath manually. (usually by putting the jar into /lib)\n+\n+You can write your own Reporter by implementing the `org.apache.flink.metrics.reporter.MetricReporter` interface.\n+If the Reporter should send out reports regularly you have to implement the Scheduled interface as well.\n+\n+Flink supports the following systems with their respective configuration parameters:\n+\n+#### Ganglia (org.apcahe.flink.metrics.ganglia.GangliaReporter)\n+- `host`\n+- `port`\n+- `dmax`\n+- `tmax`\n+- `ttl`\n+- `addressingMode` (UNICAST/MULTICAST)\n+\n+#### Graphite (org.apache.flink.metrics.graphite.GraphiteReporter)\n+- `host`\n+- `port`","path":"docs/apis/common/index.md","position":135,"original_position":135,"commit_id":"f275fc0ff26fdbf91943508bc29ce55c46b8a491","original_commit_id":"f275fc0ff26fdbf91943508bc29ce55c46b8a491","user":{"login":"tillrohrmann","id":5756858,"avatar_url":"https://avatars.githubusercontent.com/u/5756858?v=3","gravatar_id":"","url":"https://api.github.com/users/tillrohrmann","html_url":"https://github.com/tillrohrmann","followers_url":"https://api.github.com/users/tillrohrmann/followers","following_url":"https://api.github.com/users/tillrohrmann/following{/other_user}","gists_url":"https://api.github.com/users/tillrohrmann/gists{/gist_id}","starred_url":"https://api.github.com/users/tillrohrmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tillrohrmann/subscriptions","organizations_url":"https://api.github.com/users/tillrohrmann/orgs","repos_url":"https://api.github.com/users/tillrohrmann/repos","events_url":"https://api.github.com/users/tillrohrmann/events{/privacy}","received_events_url":"https://api.github.com/users/tillrohrmann/received_events","type":"User","site_admin":false},"body":"Explanation of parameters","created_at":"2016-06-27T11:58:47Z","updated_at":"2016-06-27T11:58:47Z","html_url":"https://github.com/apache/flink/pull/2158#discussion_r68564218","pull_request_url":"https://api.github.com/repos/apache/flink/pulls/2158","_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/comments/68564218"},"html":{"href":"https://github.com/apache/flink/pull/2158#discussion_r68564218"},"pull_request":{"href":"https://api.github.com/repos/apache/flink/pulls/2158"}}},"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/2158","id":75091412,"html_url":"https://github.com/apache/flink/pull/2158","diff_url":"https://github.com/apache/flink/pull/2158.diff","patch_url":"https://github.com/apache/flink/pull/2158.patch","issue_url":"https://api.github.com/repos/apache/flink/issues/2158","number":2158,"state":"open","locked":false,"title":"[FLINK-4116] Metrics documentation","user":{"login":"zentol","id":5725237,"avatar_url":"https://avatars.githubusercontent.com/u/5725237?v=3","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"body":"","created_at":"2016-06-24T10:02:32Z","updated_at":"2016-06-27T11:58:47Z","closed_at":null,"merged_at":null,"merge_commit_sha":"26af693c206676fb088f269f3c945f389cb1d62c","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/flink/pulls/2158/commits","review_comments_url":"https://api.github.com/repos/apache/flink/pulls/2158/comments","review_comment_url":"https://api.github.com/repos/apache/flink/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/flink/issues/2158/comments","statuses_url":"https://api.github.com/repos/apache/flink/statuses/f275fc0ff26fdbf91943508bc29ce55c46b8a491","head":{"label":"zentol:metrics_docs","ref":"metrics_docs","sha":"f275fc0ff26fdbf91943508bc29ce55c46b8a491","user":{"login":"zentol","id":5725237,"avatar_url":"https://avatars.githubusercontent.com/u/5725237?v=3","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"repo":{"id":20647396,"name":"flink","full_name":"zentol/flink","owner":{"login":"zentol","id":5725237,"avatar_url":"https://avatars.githubusercontent.com/u/5725237?v=3","gravatar_id":"","url":"https://api.github.com/users/zentol","html_url":"https://github.com/zentol","followers_url":"https://api.github.com/users/zentol/followers","following_url":"https://api.github.com/users/zentol/following{/other_user}","gists_url":"https://api.github.com/users/zentol/gists{/gist_id}","starred_url":"https://api.github.com/users/zentol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zentol/subscriptions","organizations_url":"https://api.github.com/users/zentol/orgs","repos_url":"https://api.github.com/users/zentol/repos","events_url":"https://api.github.com/users/zentol/events{/privacy}","received_events_url":"https://api.github.com/users/zentol/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/zentol/flink","description":"Mirror of Apache Flink","fork":true,"url":"https://api.github.com/repos/zentol/flink","forks_url":"https://api.github.com/repos/zentol/flink/forks","keys_url":"https://api.github.com/repos/zentol/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zentol/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zentol/flink/teams","hooks_url":"https://api.github.com/repos/zentol/flink/hooks","issue_events_url":"https://api.github.com/repos/zentol/flink/issues/events{/number}","events_url":"https://api.github.com/repos/zentol/flink/events","assignees_url":"https://api.github.com/repos/zentol/flink/assignees{/user}","branches_url":"https://api.github.com/repos/zentol/flink/branches{/branch}","tags_url":"https://api.github.com/repos/zentol/flink/tags","blobs_url":"https://api.github.com/repos/zentol/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zentol/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zentol/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/zentol/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zentol/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/zentol/flink/languages","stargazers_url":"https://api.github.com/repos/zentol/flink/stargazers","contributors_url":"https://api.github.com/repos/zentol/flink/contributors","subscribers_url":"https://api.github.com/repos/zentol/flink/subscribers","subscription_url":"https://api.github.com/repos/zentol/flink/subscription","commits_url":"https://api.github.com/repos/zentol/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/zentol/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/zentol/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/zentol/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/zentol/flink/contents/{+path}","compare_url":"https://api.github.com/repos/zentol/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zentol/flink/merges","archive_url":"https://api.github.com/repos/zentol/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zentol/flink/downloads","issues_url":"https://api.github.com/repos/zentol/flink/issues{/number}","pulls_url":"https://api.github.com/repos/zentol/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/zentol/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/zentol/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zentol/flink/labels{/name}","releases_url":"https://api.github.com/repos/zentol/flink/releases{/id}","deployments_url":"https://api.github.com/repos/zentol/flink/deployments","created_at":"2014-06-09T13:27:33Z","updated_at":"2016-01-19T13:55:47Z","pushed_at":"2016-06-27T11:18:18Z","git_url":"git://github.com/zentol/flink.git","ssh_url":"git@github.com:zentol/flink.git","clone_url":"https://github.com/zentol/flink.git","svn_url":"https://github.com/zentol/flink","homepage":null,"size":90672,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"6420c1c264ed3ce0c32ba164c2cdb85ccdccf265","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":20587599,"name":"flink","full_name":"apache/flink","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/flink","description":"Mirror of Apache Flink","fork":false,"url":"https://api.github.com/repos/apache/flink","forks_url":"https://api.github.com/repos/apache/flink/forks","keys_url":"https://api.github.com/repos/apache/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/flink/teams","hooks_url":"https://api.github.com/repos/apache/flink/hooks","issue_events_url":"https://api.github.com/repos/apache/flink/issues/events{/number}","events_url":"https://api.github.com/repos/apache/flink/events","assignees_url":"https://api.github.com/repos/apache/flink/assignees{/user}","branches_url":"https://api.github.com/repos/apache/flink/branches{/branch}","tags_url":"https://api.github.com/repos/apache/flink/tags","blobs_url":"https://api.github.com/repos/apache/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/flink/languages","stargazers_url":"https://api.github.com/repos/apache/flink/stargazers","contributors_url":"https://api.github.com/repos/apache/flink/contributors","subscribers_url":"https://api.github.com/repos/apache/flink/subscribers","subscription_url":"https://api.github.com/repos/apache/flink/subscription","commits_url":"https://api.github.com/repos/apache/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/flink/contents/{+path}","compare_url":"https://api.github.com/repos/apache/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/flink/merges","archive_url":"https://api.github.com/repos/apache/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/flink/downloads","issues_url":"https://api.github.com/repos/apache/flink/issues{/number}","pulls_url":"https://api.github.com/repos/apache/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/flink/labels{/name}","releases_url":"https://api.github.com/repos/apache/flink/releases{/id}","deployments_url":"https://api.github.com/repos/apache/flink/deployments","created_at":"2014-06-07T07:00:10Z","updated_at":"2016-06-27T10:11:06Z","pushed_at":"2016-06-27T11:29:25Z","git_url":"git://github.com/apache/flink.git","ssh_url":"git@github.com:apache/flink.git","clone_url":"https://github.com/apache/flink.git","svn_url":"https://github.com/apache/flink","homepage":null,"size":120241,"stargazers_count":1389,"watchers_count":1389,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":897,"mirror_url":"git://git.apache.org/flink.git","open_issues_count":118,"forks":897,"open_issues":118,"watchers":1389,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/2158"},"html":{"href":"https://github.com/apache/flink/pull/2158"},"issue":{"href":"https://api.github.com/repos/apache/flink/issues/2158"},"comments":{"href":"https://api.github.com/repos/apache/flink/issues/2158/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/flink/pulls/2158/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/flink/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/flink/pulls/2158/commits"},"statuses":{"href":"https://api.github.com/repos/apache/flink/statuses/f275fc0ff26fdbf91943508bc29ce55c46b8a491"}}}}
{"actor":{"display_login":"tillrohrmann"}}