{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/logstash/issues/4492","labels_url":"https://api.github.com/repos/elastic/logstash/issues/4492/labels{/name}","comments_url":"https://api.github.com/repos/elastic/logstash/issues/4492/comments","events_url":"https://api.github.com/repos/elastic/logstash/issues/4492/events","html_url":"https://github.com/elastic/logstash/issues/4492","id":126748871,"number":4492,"title":"Internal structure of the metric inside the metric store","user":{"login":"ph","id":640,"avatar_url":"https://avatars.githubusercontent.com/u/640?v=3","gravatar_id":"","url":"https://api.github.com/users/ph","html_url":"https://github.com/ph","followers_url":"https://api.github.com/users/ph/followers","following_url":"https://api.github.com/users/ph/following{/other_user}","gists_url":"https://api.github.com/users/ph/gists{/gist_id}","starred_url":"https://api.github.com/users/ph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ph/subscriptions","organizations_url":"https://api.github.com/users/ph/orgs","repos_url":"https://api.github.com/users/ph/repos","events_url":"https://api.github.com/users/ph/events{/privacy}","received_events_url":"https://api.github.com/users/ph/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/logstash/labels/design","name":"design","color":"eb6420"},{"url":"https://api.github.com/repos/elastic/logstash/labels/metrics","name":"metrics","color":"c7def8"}],"state":"open","locked":false,"assignee":{"login":"ph","id":640,"avatar_url":"https://avatars.githubusercontent.com/u/640?v=3","gravatar_id":"","url":"https://api.github.com/users/ph","html_url":"https://github.com/ph","followers_url":"https://api.github.com/users/ph/followers","following_url":"https://api.github.com/users/ph/following{/other_user}","gists_url":"https://api.github.com/users/ph/gists{/gist_id}","starred_url":"https://api.github.com/users/ph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ph/subscriptions","organizations_url":"https://api.github.com/users/ph/orgs","repos_url":"https://api.github.com/users/ph/repos","events_url":"https://api.github.com/users/ph/events{/privacy}","received_events_url":"https://api.github.com/users/ph/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2016-01-14T21:10:45Z","updated_at":"2016-01-15T10:56:10Z","closed_at":null,"body":"When merging the JVM and the web api we found out that the current way of defining namespace and keys wasn't flexible enough.\r\n\r\nLets try to define all all the possibilities we can have.\r\n\r\n# Top level\r\n```\r\nstats.events.in => ever increasing counter\r\nstats.events.out => ever increasing counter\r\nstats.uptime_in_milliseconds => gauge value\r\nstats.version => gauge value\r\n```\r\n```\r\nstats.jvm.memory.heap.used_in_bytes => gauge value\r\nstats.jvm.memory.used_in_bytes => gauge\r\nstats.jvm.memory.used_percent => gauge value\r\nstats.jvm.memory.committed_in_bytes => gauge\r\nstats.jvm.memory.max_in_bytes => gauge\r\n```\r\n```\r\nstats.os.cpu.XXX\r\n```\r\n# Detailed pipelines level, collected at the pipeline level\r\nIn the following case **XXX** is a **UUID** of user defined string in the logstash configuration.\r\n\r\n```\r\nstats.pipelines.pipeline_X.events.in => ever increasing counter\r\nstats.pipelines.pipeline_X.events.out => ever increasing counter\r\nstats.pipelines.pipeliens_X.plugins.input.file_XXXXXX.events.out => ever increasing counter\r\nstats.pipelines.pipelines_X.plugins.filters.grok_XXXXXX.events.out => ever increasing counter\r\nstats.pipelines.pipelines_X.plugins.filters.grok_XXXXXX.events.in => ever increasing counter\r\nstats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.events.in => ever increasing counter\r\nstats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.events.out => need to be done at the plugin level or with a ack.\r\n```\r\n# Plugins Level metrics\r\nTheses statistic are collected by the code inside the plugin and are defined by the plugin authors.\r\n```\r\nstats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.internal.error_count => ever increasing counter\r\nstats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.internal.retry_count => ever increasing counter\r\n```\r\n# Query\r\nAll the keys are queryable on the metric store and return nested information when possible.\r\nFrom a users point of vue they will be transformed into a hash.\r\n\r\n```\r\nstats.pipelines < return all the pipelines information\r\nstats.pipelines.pipeline_X < Return all the metrics from the pipeline X\r\n```\r\n\r\nFor keeping thing simple in the first iteration the store doesn't support doing wildcards query,\r\nbut this could probably be done if needed in another iteration.\r\n\r\n\r\n\r\n# API inside the pipelines or logstash core.\r\n\r\nIn all the following scenarios is recommended to use a variable when you are doing string interpolation.\r\n\r\n```ruby\r\nmetric.increment(:\"stats.events.in\", 100)\r\nmetric.increment(:\"stats.events.in\", 100)\r\nmetric.increment(:\"stats.events.out\", 45)\r\nmetric.increment(:\"stats.events.#{pipeline_id}.events.in\", 32)\r\nmetric.gauge(:\"stats.jvm.memory\", 1200)\r\n```\r\n# API inside the plugins used by dev author.\r\nPlugins authors need to be able to collect metric but we also need to scope where their metrics will be save.\r\nWhen a plugin is created we are giving the plugin instance a specific metric instance correctly scoped with the an initial namespace.\r\n\r\n```ruby\r\nnamespaced_metric = NamedSpacedMetric.new(:\"stats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.internal\", metric)\r\nnamespaced_metric.increment(:error_count, 10) # will create stats.pipelines.pipelines_X.plugins.outputs.elasticsearch_XXXX.internal.error_count\r\n```\r\n\r\n# Metric Type\r\n## Counter\r\nIn the current state of the code we only have the counter type which is a ever increasing counter.\r\nWe can use this to send data to elasticsearch and use the aggregation to retrieve the events per seconds in the system.\r\nNo calculation is done on the logstash side. So the user of the api will only see the ever increasing values, they will need to do their own calculation.\r\n\r\n## Gauge\r\nMore details in https://github.com/elastic/logstash/issues/4488\r\nThis values is mostly a snapshot in time of something, we can use this for memory usage, cpu usage and the uptime.\r\nAgain this value with the sampling will allow us to graph the evolutions over time if needed.\r\n\r\n## Response Meter\r\nWe will probably need something to collect response time, because a counter or a gauge make us lose way too much granularity.\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/logstash/issues/comments/171932852","html_url":"https://github.com/elastic/logstash/issues/4492#issuecomment-171932852","issue_url":"https://api.github.com/repos/elastic/logstash/issues/4492","id":171932852,"user":{"login":"purbon","id":68540,"avatar_url":"https://avatars.githubusercontent.com/u/68540?v=3","gravatar_id":"","url":"https://api.github.com/users/purbon","html_url":"https://github.com/purbon","followers_url":"https://api.github.com/users/purbon/followers","following_url":"https://api.github.com/users/purbon/following{/other_user}","gists_url":"https://api.github.com/users/purbon/gists{/gist_id}","starred_url":"https://api.github.com/users/purbon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/purbon/subscriptions","organizations_url":"https://api.github.com/users/purbon/orgs","repos_url":"https://api.github.com/users/purbon/repos","events_url":"https://api.github.com/users/purbon/events{/privacy}","received_events_url":"https://api.github.com/users/purbon/received_events","type":"User","site_admin":false},"created_at":"2016-01-15T10:56:10Z","updated_at":"2016-01-15T10:56:10Z","body":"on it @ph "}}