{"action":"created","issue":{"url":"https://api.github.com/repos/esl/MongooseIM/issues/1959","repository_url":"https://api.github.com/repos/esl/MongooseIM","labels_url":"https://api.github.com/repos/esl/MongooseIM/issues/1959/labels{/name}","comments_url":"https://api.github.com/repos/esl/MongooseIM/issues/1959/comments","events_url":"https://api.github.com/repos/esl/MongooseIM/issues/1959/events","html_url":"https://github.com/esl/MongooseIM/issues/1959","id":337898563,"node_id":"MDU6SXNzdWUzMzc4OTg1NjM=","number":1959,"title":"Worker pool unification, iteration 2","user":{"login":"fenek","id":548816,"node_id":"MDQ6VXNlcjU0ODgxNg==","avatar_url":"https://avatars0.githubusercontent.com/u/548816?v=4","gravatar_id":"","url":"https://api.github.com/users/fenek","html_url":"https://github.com/fenek","followers_url":"https://api.github.com/users/fenek/followers","following_url":"https://api.github.com/users/fenek/following{/other_user}","gists_url":"https://api.github.com/users/fenek/gists{/gist_id}","starred_url":"https://api.github.com/users/fenek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fenek/subscriptions","organizations_url":"https://api.github.com/users/fenek/orgs","repos_url":"https://api.github.com/users/fenek/repos","events_url":"https://api.github.com/users/fenek/events{/privacy}","received_events_url":"https://api.github.com/users/fenek/received_events","type":"User","site_admin":false},"labels":[{"id":285680951,"node_id":"MDU6TGFiZWwyODU2ODA5NTE=","url":"https://api.github.com/repos/esl/MongooseIM/labels/to-discuss","name":"to-discuss","color":"207de5","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-07-03T13:26:32Z","updated_at":"2018-07-05T10:46:07Z","closed_at":null,"author_association":"MEMBER","body":"Hi everyone,\r\n\r\nI was thinking about unifying all outgoing connection pools in terms of configuration and reliability. Bartek Gorny already did the unified supervision and access parts. :)\r\n\r\n# Configuration\r\n\r\n## Schema\r\n\r\nCurrent configuration model would be replaced with following schema:\r\n\r\n```erlang\r\n{outgoing_pools, [\r\n  {redis, host, default, PoolOpts, ConnOpts},\r\n  {rdbms, host, read, PoolOpts, ConnOpts},\r\n  {rdbms, host, [write, default], PoolOpts, ConnOpts},\r\n  {riak, global, default, PoolOpts, ConnOpts},\r\n  {cassandra, global, default, PoolOpts, ConnOpts},\r\n  {elasticsearch, global, default, PoolOpts, ConnOpts},\r\n  {http, host, auth, PoolOpts, ConnOpts},\r\n  {http, global, default, PoolOpts, ConnOpts}\r\n]}.\r\n```\r\n\r\n* First tuple element is the pool type. Used by `mongoose_wpool` to automatically call proper `mongoose_wpool_[type]` modules.\r\n* Second element states whether one pool should be created per vhost or per whole node.\r\n* Third element is the pool tag. Could be any atom with some limitations:\r\n  * `default` - default (:)) pool used by various modules if not specified otherwise.\r\n  * `read` and `write` are not reserved but by convention would be used to separate types of operations by various modules, e.g. `mod_roster`.\r\n* `PoolOpts` are the pool parameters (size, strategy etc.)\r\n* `ConnOpts` are the parameters used by the connection library itself.\r\n\r\n## Pool choice strategy\r\n\r\nFor now modules will expect precise configuration of pools. It means that if an admin configures mod_roster to use `{global, default}` for writes and `{host, read}` for reads, no other pools will be attempted automatically (could be a nice feature in future iterations though).\r\n\r\n# Automatic reconnection\r\n\r\nWorker pool should automatically perform reconnections with various backoff strategies. Reconnection events would be recorded in metrics. Reconnection mechanism in actual connection library, if present, should be disabled.\r\n\r\n# Misc\r\n\r\n## Consistent naming scheme\r\n\r\nAll pools will have following names:\r\n\r\n`mongoose_wpool_[type]_[scope]_[tag]` where `[scope]` is `global` or specific hostname.\r\n\r\n## Metrics\r\n\r\n`mongoose_wpool` must provide a default set of metrics for every running pool:\r\n\r\nTheir names would be `[global | Host, mongoose_wpool, type, Metric]`.\r\n\r\nFollowing stats should be exposed:\r\n\r\n* `size`\r\n* `total_message_queue_len`\r\n* `connections` - bumped by successful reconnection\r\n* `disconnections` - not bumped by failed connection\r\n* `failed_connections` - bumped by both ordinary connection and reconnection failure\r\n* `reconnection_attempts`\r\n* `successful_requests`\r\n* `failed_requests`\r\n\r\n## Config reload\r\n\r\n`mongoose_wpool` will expose a function that `ejabberd_config` will call upon config reload. It will delegate a resposibility of closing/opening proper pools in a smart way (no crude stop & start of everything).\r\n\r\n## supervisor2\r\n\r\nsupervisor2 would become obsolete, as its main benefit is an option of endless restarts with some cooldown."},"comment":{"url":"https://api.github.com/repos/esl/MongooseIM/issues/comments/402682883","html_url":"https://github.com/esl/MongooseIM/issues/1959#issuecomment-402682883","issue_url":"https://api.github.com/repos/esl/MongooseIM/issues/1959","id":402682883,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMjY4Mjg4Mw==","user":{"login":"michalwski","id":705828,"node_id":"MDQ6VXNlcjcwNTgyOA==","avatar_url":"https://avatars3.githubusercontent.com/u/705828?v=4","gravatar_id":"","url":"https://api.github.com/users/michalwski","html_url":"https://github.com/michalwski","followers_url":"https://api.github.com/users/michalwski/followers","following_url":"https://api.github.com/users/michalwski/following{/other_user}","gists_url":"https://api.github.com/users/michalwski/gists{/gist_id}","starred_url":"https://api.github.com/users/michalwski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michalwski/subscriptions","organizations_url":"https://api.github.com/users/michalwski/orgs","repos_url":"https://api.github.com/users/michalwski/repos","events_url":"https://api.github.com/users/michalwski/events{/privacy}","received_events_url":"https://api.github.com/users/michalwski/received_events","type":"User","site_admin":false},"created_at":"2018-07-05T10:46:07Z","updated_at":"2018-07-05T10:46:07Z","author_association":"MEMBER","body":"Yes it does! Good idea."}}
{"actor":{"display_login":"michalwski"}}