{"action":"created","issue":{"url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/226","repository_url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease","labels_url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/226/labels{/name}","comments_url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/226/comments","events_url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/226/events","html_url":"https://github.com/bosh-prometheus/prometheus-boshrelease/issues/226","id":340185095,"node_id":"MDU6SXNzdWUzNDAxODUwOTU=","number":226,"title":"Refine Cf Cloud Controller dashboard & alerts with more capacity-related metrics","user":{"login":"gberche-orange","id":4748380,"node_id":"MDQ6VXNlcjQ3NDgzODA=","avatar_url":"https://avatars1.githubusercontent.com/u/4748380?v=4","gravatar_id":"","url":"https://api.github.com/users/gberche-orange","html_url":"https://github.com/gberche-orange","followers_url":"https://api.github.com/users/gberche-orange/followers","following_url":"https://api.github.com/users/gberche-orange/following{/other_user}","gists_url":"https://api.github.com/users/gberche-orange/gists{/gist_id}","starred_url":"https://api.github.com/users/gberche-orange/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gberche-orange/subscriptions","organizations_url":"https://api.github.com/users/gberche-orange/orgs","repos_url":"https://api.github.com/users/gberche-orange/repos","events_url":"https://api.github.com/users/gberche-orange/events{/privacy}","received_events_url":"https://api.github.com/users/gberche-orange/received_events","type":"User","site_admin":false},"labels":[{"id":202484666,"node_id":"MDU6TGFiZWwyMDI0ODQ2NjY=","url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/labels/help%20wanted","name":"help wanted","color":"159818","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-07-11T10:34:13Z","updated_at":"2018-07-31T11:53:02Z","closed_at":null,"author_association":"NONE","body":"We recently experienced an outage related to our blobstore (minio) capacity, which was not coping fast enough with diego requests to download droplets (in the scenario of a cold start of the whole CF deployment). As a result, this created a capacity issue on the cloudcontroller whose 20 threads started getting all busy on blobstore responses, and the API server began being slow, and being restarted by their health checks not completing within timeout.\r\n\r\nThe [CC capacity-related metrics](https://docs.cloudfoundry.org/running/all_metrics.html#cc) such as ``thread_info.event_machine.threadqueue.size`` (number of unscheduled tasks in the threadqueue)  would be useful to diagnose such CC capacity issues, along with a related alert.\r\n\r\n![image](https://user-images.githubusercontent.com/4748380/42566146-ea1a7de0-8505-11e8-8ea4-4b3c1910c318.png)\r\n"},"comment":{"url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/comments/409193394","html_url":"https://github.com/bosh-prometheus/prometheus-boshrelease/issues/226#issuecomment-409193394","issue_url":"https://api.github.com/repos/bosh-prometheus/prometheus-boshrelease/issues/226","id":409193394,"node_id":"MDEyOklzc3VlQ29tbWVudDQwOTE5MzM5NA==","user":{"login":"gberche-orange","id":4748380,"node_id":"MDQ6VXNlcjQ3NDgzODA=","avatar_url":"https://avatars1.githubusercontent.com/u/4748380?v=4","gravatar_id":"","url":"https://api.github.com/users/gberche-orange","html_url":"https://github.com/gberche-orange","followers_url":"https://api.github.com/users/gberche-orange/followers","following_url":"https://api.github.com/users/gberche-orange/following{/other_user}","gists_url":"https://api.github.com/users/gberche-orange/gists{/gist_id}","starred_url":"https://api.github.com/users/gberche-orange/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gberche-orange/subscriptions","organizations_url":"https://api.github.com/users/gberche-orange/orgs","repos_url":"https://api.github.com/users/gberche-orange/repos","events_url":"https://api.github.com/users/gberche-orange/events{/privacy}","received_events_url":"https://api.github.com/users/gberche-orange/received_events","type":"User","site_admin":false},"created_at":"2018-07-31T11:53:02Z","updated_at":"2018-07-31T11:53:02Z","author_association":"NONE","body":"@frodenas we've added that to our backlog (see linked issue in paas-templates repo), but I can't tell how soon we'll be able to tackle this."}}
{"actor":{"display_login":"gberche-orange"}}