{"action":"created","issue":{"url":"https://api.github.com/repos/getsentry/sentry/issues/2693","repository_url":"https://api.github.com/repos/getsentry/sentry","labels_url":"https://api.github.com/repos/getsentry/sentry/issues/2693/labels{/name}","comments_url":"https://api.github.com/repos/getsentry/sentry/issues/2693/comments","events_url":"https://api.github.com/repos/getsentry/sentry/issues/2693/events","html_url":"https://github.com/getsentry/sentry/issues/2693","id":134124195,"number":2693,"title":"Restructure Redis connection management.","user":{"login":"tkaemming","id":65315,"avatar_url":"https://avatars.githubusercontent.com/u/65315?v=3","gravatar_id":"","url":"https://api.github.com/users/tkaemming","html_url":"https://github.com/tkaemming","followers_url":"https://api.github.com/users/tkaemming/followers","following_url":"https://api.github.com/users/tkaemming/following{/other_user}","gists_url":"https://api.github.com/users/tkaemming/gists{/gist_id}","starred_url":"https://api.github.com/users/tkaemming/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tkaemming/subscriptions","organizations_url":"https://api.github.com/users/tkaemming/orgs","repos_url":"https://api.github.com/users/tkaemming/repos","events_url":"https://api.github.com/users/tkaemming/events{/privacy}","received_events_url":"https://api.github.com/users/tkaemming/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"tkaemming","id":65315,"avatar_url":"https://avatars.githubusercontent.com/u/65315?v=3","gravatar_id":"","url":"https://api.github.com/users/tkaemming","html_url":"https://github.com/tkaemming","followers_url":"https://api.github.com/users/tkaemming/followers","following_url":"https://api.github.com/users/tkaemming/following{/other_user}","gists_url":"https://api.github.com/users/tkaemming/gists{/gist_id}","starred_url":"https://api.github.com/users/tkaemming/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tkaemming/subscriptions","organizations_url":"https://api.github.com/users/tkaemming/orgs","repos_url":"https://api.github.com/users/tkaemming/repos","events_url":"https://api.github.com/users/tkaemming/events{/privacy}","received_events_url":"https://api.github.com/users/tkaemming/received_events","type":"User","site_admin":false},"milestone":null,"comments":4,"created_at":"2016-02-16T23:16:40Z","updated_at":"2016-02-17T00:25:11Z","closed_at":null,"body":"# Overview\r\n\r\nAdd `SENTRY_REDIS_CLUSTERS` to settings (and an equivalent value to `sentry.options`), replacing `SENTRY_REDIS_OPTIONS`. This is similar to the `DATABASES` setting, except values are a map that will be passed to the [`Cluster` constructor](https://github.com/getsentry/rb/blob/50ede887d9980d46fbe52aaa32d884c425f29c18/rb/cluster.py#L89).\r\n\r\nFor example, the current default:\r\n\r\n```\r\nSENTRY_REDIS_OPTIONS = {\r\n    'hosts': {\r\n        0: {\r\n            'host': '127.0.0.1',\r\n            'port': 6379,\r\n        }\r\n    }\r\n}\r\n```\r\n\r\nwould become:\r\n\r\n```\r\nSENTRY_REDIS_CLUSTERS = {\r\n    'default':\r\n        'hosts': {\r\n            0: {\r\n                'host': '127.0.0.1',\r\n                'port': 6379,\r\n            }\r\n        }\r\n    },\r\n}\r\n```\r\n\r\nIn backends, the target cluster(s) should be provided by name. If no cluster is provided, either use the default cluster, or fall back to the old method of configuration (providing options directly to the backend constructor) and throw a deprecation warning. Don't allow passing both the cluster and old parameters.\r\n\r\nFor third-party plugin compatibility, create `SENTRY_REDIS_OPTIONS` from the default cluster connection options. This will be removed in the future, since it circumvents [our factory method that uses a shared connection pool](https://github.com/getsentry/sentry/blob/6a4227f32543d459e859c3fd8eb64b0ccb697be7/src/sentry/utils/redis.py#L18-L50). We'll need to decide on when formally remove this.\r\n\r\nIt'd also be nice to create `BROKER_URL` from the default cluster connection options if none is provided. (Maybe not?)\r\n\r\n# Tasks\r\n\r\n- [ ] Add `SENTRY_REDIS_CLUSTERS` to default settings template in [`sentry/runner/settings.py`](https://github.com/getsentry/sentry/blob/65332fc9ead7dafdb75eedeb672dbe1eabf4543d/src/sentry/runner/settings.py).\r\n- [ ] Add backwards compatibility layer to allow access to `SENTRY_REDIS_OPTIONS` from the derived value from `SENTRY_REDIS_CLUSTERS`.\r\n- [ ] Add `redis.clusters` to [`sentry.options`](https://github.com/getsentry/sentry/blob/65332fc9ead7dafdb75eedeb672dbe1eabf4543d/src/sentry/options/defaults.py).\r\n- [ ] Deprecate the [`redis.options` option](https://github.com/getsentry/sentry/blob/65332fc9ead7dafdb75eedeb672dbe1eabf4543d/src/sentry/options/defaults.py#L22), and add backwards compatibility later to allow accessing the default cluster via this option, similar to settings.\r\n- [ ] Figure out a formal deprecation strategy for `SENTRY_REDIS_OPTIONS`.\r\n- [ ] Add Redis cluster manager that allows retrieving clusters by name, and uses our shared connection pool.\r\n- [ ] Update backends to use named clusters rather than connection parameters.\r\n  - [ ] Buffers\r\n  - [ ] Cache\r\n  - [ ] Digests\r\n  - [ ] Quotas\r\n  - [ ] Rate Limiter\r\n  - [ ] TSDB\r\n- [ ] Update supported plugins.\r\n- [ ] Update documentation to reference `SENTRY_REDIS_CLUSTERS` instead of `SENTRY_REDIS_OPTIONS`. (Should we just remove mentioning settings entirely in the future, and only refer to the configuration file?)"},"comment":{"url":"https://api.github.com/repos/getsentry/sentry/issues/comments/184938954","html_url":"https://github.com/getsentry/sentry/issues/2693#issuecomment-184938954","issue_url":"https://api.github.com/repos/getsentry/sentry/issues/2693","id":184938954,"user":{"login":"tkaemming","id":65315,"avatar_url":"https://avatars.githubusercontent.com/u/65315?v=3","gravatar_id":"","url":"https://api.github.com/users/tkaemming","html_url":"https://github.com/tkaemming","followers_url":"https://api.github.com/users/tkaemming/followers","following_url":"https://api.github.com/users/tkaemming/following{/other_user}","gists_url":"https://api.github.com/users/tkaemming/gists{/gist_id}","starred_url":"https://api.github.com/users/tkaemming/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tkaemming/subscriptions","organizations_url":"https://api.github.com/users/tkaemming/orgs","repos_url":"https://api.github.com/users/tkaemming/repos","events_url":"https://api.github.com/users/tkaemming/events{/privacy}","received_events_url":"https://api.github.com/users/tkaemming/received_events","type":"User","site_admin":false},"created_at":"2016-02-17T00:25:11Z","updated_at":"2016-02-17T00:25:11Z","body":"Also, does it make sense to remove the Redis configurations from the settings file (other than `BROKER_URL` or anything necessary for temporary backwards compatibility) and allow \"new style\" named cluster configuration to only happen via the configuration file?"}}