{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/kibana/issues/18573","repository_url":"https://api.github.com/repos/elastic/kibana","labels_url":"https://api.github.com/repos/elastic/kibana/issues/18573/labels{/name}","comments_url":"https://api.github.com/repos/elastic/kibana/issues/18573/comments","events_url":"https://api.github.com/repos/elastic/kibana/issues/18573/events","html_url":"https://github.com/elastic/kibana/pull/18573","id":317710691,"number":18573,"title":"[PipelineViewer] [Monitoring] Add Queue Type agg to pipeline stats","user":{"login":"justinkambic","id":18429259,"avatar_url":"https://avatars2.githubusercontent.com/u/18429259?v=4","gravatar_id":"","url":"https://api.github.com/users/justinkambic","html_url":"https://github.com/justinkambic","followers_url":"https://api.github.com/users/justinkambic/followers","following_url":"https://api.github.com/users/justinkambic/following{/other_user}","gists_url":"https://api.github.com/users/justinkambic/gists{/gist_id}","starred_url":"https://api.github.com/users/justinkambic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinkambic/subscriptions","organizations_url":"https://api.github.com/users/justinkambic/orgs","repos_url":"https://api.github.com/users/justinkambic/repos","events_url":"https://api.github.com/users/justinkambic/events{/privacy}","received_events_url":"https://api.github.com/users/justinkambic/received_events","type":"User","site_admin":false},"labels":[{"id":832655738,"url":"https://api.github.com/repos/elastic/kibana/labels/:Logstash","name":":Logstash","color":"bfdadc","default":false},{"id":912940115,"url":"https://api.github.com/repos/elastic/kibana/labels/:Monitoring","name":":Monitoring","color":"fef2c0","default":false},{"id":135307740,"url":"https://api.github.com/repos/elastic/kibana/labels/review","name":"review","color":"0052cc","default":false},{"id":883958396,"url":"https://api.github.com/repos/elastic/kibana/labels/v6.4.0","name":"v6.4.0","color":"ffffff","default":false},{"id":654061976,"url":"https://api.github.com/repos/elastic/kibana/labels/v7.0.0","name":"v7.0.0","color":"ffffff","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-04-25T16:52:01Z","updated_at":"2018-04-25T20:57:31Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/elastic/kibana/pulls/18573","html_url":"https://github.com/elastic/kibana/pull/18573","diff_url":"https://github.com/elastic/kibana/pull/18573.diff","patch_url":"https://github.com/elastic/kibana/pull/18573.patch"},"body":"## Changes\r\nThe goal here is to make it possible for the Pipeline Viewer's graph model classes to hold data about the type of queue in use by a pipeline.\r\n\r\nThe aggregation I've added will use the same timeseries/bucketing as the other stats aggregations that are added to the `LogstashState` document that we return for a pipeline request. For each State, this code references the vertex for the Pipeline's queue, adds a new array called `typeStats`, and then pushes an object per queue type, along with the associated `doc_count`, for each time series bucket. The client classes can then consume or manipulate the `typeStats` to display queue type information like they do for other metric or usage information.\r\n\r\nThe variable/key names are WIP, I wanted to get other eyes on this from a functional standpoint before I spend any more time going down this path.\r\n\r\nEDIT: to help with visualization, here's example output from the updates to the ES aggs/query\r\n```\r\n...\r\n  \"aggregations\": {\r\n    \"by_time\": {\r\n      \"after_key\": {...},\r\n      \"buckets\": [\r\n        {\r\n          \"key\": {\r\n            \"time_bucket\": 1524582000000\r\n          },\r\n          \"doc_count\": 138,\r\n          \"pipelines\": {\r\n            \"doc_count\": 138,\r\n            \"scoped\": {\r\n              \"doc_count\": 138,\r\n              \"queue_type\": {\r\n                \"doc_count_error_upper_bound\": 0,\r\n                \"sum_other_doc_count\": 0,\r\n                \"buckets\": [\r\n                  {\r\n                    \"key\": \"persisted\",\r\n                    \"doc_count\": 138\r\n                  }\r\n                ]\r\n              },\r\n```\r\n\r\n...and the resultant object that is processed on the client as a result\r\n```\r\n{\r\n  \"id\": \"__QUEUE__\",\r\n  \"meta\": null,\r\n  \"explicit_id\": false,\r\n  \"type\": \"queue\",\r\n  \"stats\": {},\r\n  \"typeStats\": [\r\n    {\r\n      \"timestamp\": 1523973600000,\r\n      \"types\": [\r\n        {\r\n          \"type\": \"memory\",\r\n          \"count\": 92\r\n        }\r\n      ]\r\n    },\r\n    {\r\n      \"timestamp\": 1523980800000,\r\n      \"types\": [\r\n        {\r\n          \"type\": \"memory\",\r\n          \"count\": 75\r\n        }\r\n      ]\r\n    },\r\n    {\r\n      \"timestamp\": 1523984400000,\r\n      \"types\": [\r\n        {\r\n          \"type\": \"memory\",\r\n          \"count\": 360\r\n        }\r\n      ]\r\n    },\r\n...\r\n```\r\nnote - the `doc_count` values in these examples aren't supposed to match up, they're from separate queries."},"comment":{"url":"https://api.github.com/repos/elastic/kibana/issues/comments/384431612","html_url":"https://github.com/elastic/kibana/pull/18573#issuecomment-384431612","issue_url":"https://api.github.com/repos/elastic/kibana/issues/18573","id":384431612,"user":{"login":"ycombinator","id":51061,"avatar_url":"https://avatars2.githubusercontent.com/u/51061?v=4","gravatar_id":"","url":"https://api.github.com/users/ycombinator","html_url":"https://github.com/ycombinator","followers_url":"https://api.github.com/users/ycombinator/followers","following_url":"https://api.github.com/users/ycombinator/following{/other_user}","gists_url":"https://api.github.com/users/ycombinator/gists{/gist_id}","starred_url":"https://api.github.com/users/ycombinator/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ycombinator/subscriptions","organizations_url":"https://api.github.com/users/ycombinator/orgs","repos_url":"https://api.github.com/users/ycombinator/repos","events_url":"https://api.github.com/users/ycombinator/events{/privacy}","received_events_url":"https://api.github.com/users/ycombinator/received_events","type":"User","site_admin":false},"created_at":"2018-04-25T20:57:31Z","updated_at":"2018-04-25T20:57:31Z","author_association":"CONTRIBUTOR","body":"> Alternatively we could shelve the queue type addition for now. I'd prefer to give the type to the client if possible, especially as it concerns Config View and giving the user something useful to see in the Queue Statement box.\r\n\r\nWell, the queue vertex box in the current PV is also empty so I don't think the Config View would be introducing a regression. So I'd prefer to continue the discussion here while, in parallel, making progress on the Config View. Worst case we'd merge the Config View PR first and then this PR (with whatever path we end up choosing on it)."}}
{"actor":{"display_login":"ycombinator"}}