{"action":"created","issue":{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/1340","repository_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard","labels_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/1340/labels{/name}","comments_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/1340/comments","events_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/1340/events","html_url":"https://github.com/apinf/api-umbrella-dashboard/issues/1340","id":167017438,"number":1340,"title":"Benchmark Dashboard code and discuss/document possible performance improvements","user":{"login":"philippeluickx","id":836866,"avatar_url":"https://avatars.githubusercontent.com/u/836866?v=3","gravatar_id":"","url":"https://api.github.com/users/philippeluickx","html_url":"https://github.com/philippeluickx","followers_url":"https://api.github.com/users/philippeluickx/followers","following_url":"https://api.github.com/users/philippeluickx/following{/other_user}","gists_url":"https://api.github.com/users/philippeluickx/gists{/gist_id}","starred_url":"https://api.github.com/users/philippeluickx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philippeluickx/subscriptions","organizations_url":"https://api.github.com/users/philippeluickx/orgs","repos_url":"https://api.github.com/users/philippeluickx/repos","events_url":"https://api.github.com/users/philippeluickx/events{/privacy}","received_events_url":"https://api.github.com/users/philippeluickx/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/labels/Dashboard%201.0","name":"Dashboard 1.0","color":"3E4B9E"},{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/labels/in%20progress","name":"in progress","color":"ededed"},{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/labels/User%20Experience","name":"User Experience","color":"009800"}],"state":"open","locked":false,"assignee":{"login":"frenchbread","id":2122679,"avatar_url":"https://avatars.githubusercontent.com/u/2122679?v=3","gravatar_id":"","url":"https://api.github.com/users/frenchbread","html_url":"https://github.com/frenchbread","followers_url":"https://api.github.com/users/frenchbread/followers","following_url":"https://api.github.com/users/frenchbread/following{/other_user}","gists_url":"https://api.github.com/users/frenchbread/gists{/gist_id}","starred_url":"https://api.github.com/users/frenchbread/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/frenchbread/subscriptions","organizations_url":"https://api.github.com/users/frenchbread/orgs","repos_url":"https://api.github.com/users/frenchbread/repos","events_url":"https://api.github.com/users/frenchbread/events{/privacy}","received_events_url":"https://api.github.com/users/frenchbread/received_events","type":"User","site_admin":false},"assignees":[{"login":"frenchbread","id":2122679,"avatar_url":"https://avatars.githubusercontent.com/u/2122679?v=3","gravatar_id":"","url":"https://api.github.com/users/frenchbread","html_url":"https://github.com/frenchbread","followers_url":"https://api.github.com/users/frenchbread/followers","following_url":"https://api.github.com/users/frenchbread/following{/other_user}","gists_url":"https://api.github.com/users/frenchbread/gists{/gist_id}","starred_url":"https://api.github.com/users/frenchbread/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/frenchbread/subscriptions","organizations_url":"https://api.github.com/users/frenchbread/orgs","repos_url":"https://api.github.com/users/frenchbread/repos","events_url":"https://api.github.com/users/frenchbread/events{/privacy}","received_events_url":"https://api.github.com/users/frenchbread/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/milestones/28","html_url":"https://github.com/apinf/api-umbrella-dashboard/milestones/Sprint%2028","labels_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/milestones/28/labels","id":1915268,"number":28,"title":"Sprint 28","description":"Apinf sprint 28: 1 - 12 August 2016","creator":{"login":"bajiat","id":11626388,"avatar_url":"https://avatars.githubusercontent.com/u/11626388?v=3","gravatar_id":"","url":"https://api.github.com/users/bajiat","html_url":"https://github.com/bajiat","followers_url":"https://api.github.com/users/bajiat/followers","following_url":"https://api.github.com/users/bajiat/following{/other_user}","gists_url":"https://api.github.com/users/bajiat/gists{/gist_id}","starred_url":"https://api.github.com/users/bajiat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bajiat/subscriptions","organizations_url":"https://api.github.com/users/bajiat/orgs","repos_url":"https://api.github.com/users/bajiat/repos","events_url":"https://api.github.com/users/bajiat/events{/privacy}","received_events_url":"https://api.github.com/users/bajiat/received_events","type":"User","site_admin":false},"open_issues":8,"closed_issues":9,"state":"open","created_at":"2016-08-01T09:26:33Z","updated_at":"2016-08-16T13:56:54Z","due_on":"2016-08-11T21:00:00Z","closed_at":null},"comments":8,"created_at":"2016-07-22T10:33:22Z","updated_at":"2016-08-17T08:50:43Z","closed_at":null,"body":"Current Dashboard performance is slow - loading is capped at 50K records and this query take 10s. There can be several places where Dashboard performance could be improved, so care should be taken before proceeding with any single solution.\r\n## Objective\r\nBefore going with further experiments, it is good to have a baseline measure of performance for comparison. \r\n\r\nWe should try to measure the overall time, on average, that the dashboard takes to render as well as time spent on each stage. This will give us better insight on strategies for optimization.\r\n\r\n## Rendering stages\r\nThere are a few stages involved in rendering the dashboard charts, including:\r\n\r\n- **fetching** the data\r\n  - server fetch from Elastic\r\n  - client fetch from Meteor server\r\n- **parsing** the data\r\n  - e.g. aggregation or preparation of chart dimensions\r\n- **rendering** the charts\r\n  - e.g. creating DOM elements for data points, animations, etc\r\n\r\nEach stage in the rendering sequence adds some amount of delay to the overall user experience.\r\n\r\n## Proposed changes:\r\n- Aggregate data per day / response code / response time group (adds a count column)\r\n- Remove data table (can be handled in different ways later: seperate request, download CSV, in a different view,...)\r\n- Remove hourly - daily - weekly - monthly aggregation options\r\n\r\nNext steps:\r\n1. Proof of concept: only load first chart with aggregation on Elastic Search (365 records for 1 y)\r\n2. Add response code aggregation  (365 * 5 records for 1 y)\r\n3. Add response time aggregation  (365 * 5 * 10 records for 1 y) - we can group response time in 10 sections (0 - 100 - ... - +1000 ms)\r\n\r\n# Definition of done\r\n- [ ] Benchmark(s) exist(s) for Dasboard code\r\n  - [ ] Team understands, after discussion, sources of significant delay in Dashboard loading sequence\r\n- [ ] Document exists describing possible enhancements for Dashboard code\r\n  - [ ] Proposals have been objectively compared, \r\n    - e.g. honest assessment of possible delays with each solution\r\n  - [ ] Proposals have been compared by estimated amount of implementation effort\r\n  - [ ] Document acknowledges any functional losses for each proposal (e.g. dis-integration of dashboard charts), as tradeoffs"},"comment":{"url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/comments/240352228","html_url":"https://github.com/apinf/api-umbrella-dashboard/issues/1340#issuecomment-240352228","issue_url":"https://api.github.com/repos/apinf/api-umbrella-dashboard/issues/1340","id":240352228,"user":{"login":"brylie","id":17307,"avatar_url":"https://avatars.githubusercontent.com/u/17307?v=3","gravatar_id":"","url":"https://api.github.com/users/brylie","html_url":"https://github.com/brylie","followers_url":"https://api.github.com/users/brylie/followers","following_url":"https://api.github.com/users/brylie/following{/other_user}","gists_url":"https://api.github.com/users/brylie/gists{/gist_id}","starred_url":"https://api.github.com/users/brylie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brylie/subscriptions","organizations_url":"https://api.github.com/users/brylie/orgs","repos_url":"https://api.github.com/users/brylie/repos","events_url":"https://api.github.com/users/brylie/events{/privacy}","received_events_url":"https://api.github.com/users/brylie/received_events","type":"User","site_admin":false},"created_at":"2016-08-17T08:50:42Z","updated_at":"2016-08-17T08:50:42Z","body":"@philippeluickx and @frenchbread, I opened an [issue related to the API Umbrella Elasticsearch server being publicly accessible](https://github.com/apinf/api-umbrella-dashboard/issues/1402)."}}
{"actor":{"display_login":"brylie"}}