{"action":"created","issue":{"url":"https://api.github.com/repos/alphagov/paas-cf/issues/1235","repository_url":"https://api.github.com/repos/alphagov/paas-cf","labels_url":"https://api.github.com/repos/alphagov/paas-cf/issues/1235/labels{/name}","comments_url":"https://api.github.com/repos/alphagov/paas-cf/issues/1235/comments","events_url":"https://api.github.com/repos/alphagov/paas-cf/issues/1235/events","html_url":"https://github.com/alphagov/paas-cf/pull/1235","id":298280881,"number":1235,"title":"[#155178552] Monitor Logsearch Redis queue length","user":{"login":"bandesz","id":868104,"avatar_url":"https://avatars0.githubusercontent.com/u/868104?v=4","gravatar_id":"","url":"https://api.github.com/users/bandesz","html_url":"https://github.com/bandesz","followers_url":"https://api.github.com/users/bandesz/followers","following_url":"https://api.github.com/users/bandesz/following{/other_user}","gists_url":"https://api.github.com/users/bandesz/gists{/gist_id}","starred_url":"https://api.github.com/users/bandesz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bandesz/subscriptions","organizations_url":"https://api.github.com/users/bandesz/orgs","repos_url":"https://api.github.com/users/bandesz/repos","events_url":"https://api.github.com/users/bandesz/events{/privacy}","received_events_url":"https://api.github.com/users/bandesz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-02-19T13:47:35Z","updated_at":"2018-02-19T16:31:44Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/alphagov/paas-cf/pulls/1235","html_url":"https://github.com/alphagov/paas-cf/pull/1235","diff_url":"https://github.com/alphagov/paas-cf/pull/1235.diff","patch_url":"https://github.com/alphagov/paas-cf/pull/1235.patch"},"body":"## What\r\n\r\nWe want to monitor the queue length in Redis as the ingestor will start to drop messages if the queue reaches 1 million items [1]. We want to be alerted before this happens and manually intervene if necessary.\r\n\r\n[1] https://github.com/logsearch/logsearch-boshrelease/blob/v203.0.0/jobs/ingestor_syslog/spec#L61-L63\r\n\r\n❗️ This PR contains a temporary commit, please update the `datadog-for-cloudfoundry` Bosh release params after \r\n\r\n## How to review\r\n\r\n1. Deploy your CF from this branch and enable Datadog in `paas-bootstrap` and `paas-cf`:\r\n```\r\npaas-bootstrap$ make dev deployer-concourse pipelines DEPLOY_ENV=hector BRANCH=$(git rev-parse --abbrev-ref HEAD)  SELF_UPDATE_PIPELINE=false ENABLE_DATADOG=true \r\n\r\npaas-cf$ make dev pipelines BRANCH=$(git rev-parse --abbrev-ref HEAD)  SELF_UPDATE_PIPELINE=false DEPLOY_ENV=hector ENABLE_DATADOG=true\r\n```\r\n\r\n2. Go to Datadog and search for the `Logsearch Redis queue length` monitor. Check if it exists, has data and the settings make sense.\r\n\r\nOR if you are lazy, check the deployment from `DEPLOY_ENV=hector`: https://deployer.hector.dev.cloudpipeline.digital/teams/main/pipelines/create-cloudfoundry/jobs/cf-deploy\r\n\r\n\r\n## How to release\r\n\r\n1. https://github.com/alphagov/paas-release-ci/pull/55 was merged already - it fixed a Bosh release naming issue for the dev releases.\r\n1. Review and merge https://github.com/alphagov/paas-release-ci/pull/56 - it fixes a Bosh release naming issue for the final releases.\r\n1. Merge https://github.com/alphagov/paas-datadog-for-cloudfoundry-boshrelease/pull/21\r\n1. Replace the temporary commit with the final `datadog-for-cloudfoundry` release\r\n1. Merge this PR\r\n\r\n## Who can review\r\n\r\nNot @keymon or me."},"comment":{"url":"https://api.github.com/repos/alphagov/paas-cf/issues/comments/366743773","html_url":"https://github.com/alphagov/paas-cf/pull/1235#issuecomment-366743773","issue_url":"https://api.github.com/repos/alphagov/paas-cf/issues/1235","id":366743773,"user":{"login":"keymon","id":280032,"avatar_url":"https://avatars0.githubusercontent.com/u/280032?v=4","gravatar_id":"","url":"https://api.github.com/users/keymon","html_url":"https://github.com/keymon","followers_url":"https://api.github.com/users/keymon/followers","following_url":"https://api.github.com/users/keymon/following{/other_user}","gists_url":"https://api.github.com/users/keymon/gists{/gist_id}","starred_url":"https://api.github.com/users/keymon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keymon/subscriptions","organizations_url":"https://api.github.com/users/keymon/orgs","repos_url":"https://api.github.com/users/keymon/repos","events_url":"https://api.github.com/users/keymon/events{/privacy}","received_events_url":"https://api.github.com/users/keymon/received_events","type":"User","site_admin":false},"created_at":"2018-02-19T16:31:44Z","updated_at":"2018-02-19T16:31:44Z","author_association":"CONTRIBUTOR","body":"@bandesz @samcrang the datadog agent [is an addon in paas-bootstrap](https://github.com/alphagov/paas-bootstrap/blob/master/manifests/runtime-config/addons/datadog-agent.yml), you must push pipelines of paas-bootstrap with `ENABLE_DATADOG=true ` and rerun the pipeline so that the runtime config gets updated and the datadog agent gets installed."}}
{"actor":{"display_login":"keymon"}}