{"action":"closed","issue":{"url":"https://api.github.com/repos/rabbitmq/rabbitmq-server/issues/1769","repository_url":"https://api.github.com/repos/rabbitmq/rabbitmq-server","labels_url":"https://api.github.com/repos/rabbitmq/rabbitmq-server/issues/1769/labels{/name}","comments_url":"https://api.github.com/repos/rabbitmq/rabbitmq-server/issues/1769/comments","events_url":"https://api.github.com/repos/rabbitmq/rabbitmq-server/issues/1769/events","html_url":"https://github.com/rabbitmq/rabbitmq-server/issues/1769","id":381587926,"node_id":"MDU6SXNzdWUzODE1ODc5MjY=","number":1769,"title":"Queues consuming more memory in one of the node of a 3 node cluster","user":{"login":"vvrpradeep","id":15726141,"node_id":"MDQ6VXNlcjE1NzI2MTQx","avatar_url":"https://avatars3.githubusercontent.com/u/15726141?v=4","gravatar_id":"","url":"https://api.github.com/users/vvrpradeep","html_url":"https://github.com/vvrpradeep","followers_url":"https://api.github.com/users/vvrpradeep/followers","following_url":"https://api.github.com/users/vvrpradeep/following{/other_user}","gists_url":"https://api.github.com/users/vvrpradeep/gists{/gist_id}","starred_url":"https://api.github.com/users/vvrpradeep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vvrpradeep/subscriptions","organizations_url":"https://api.github.com/users/vvrpradeep/orgs","repos_url":"https://api.github.com/users/vvrpradeep/repos","events_url":"https://api.github.com/users/vvrpradeep/events{/privacy}","received_events_url":"https://api.github.com/users/vvrpradeep/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-11-16T12:54:38Z","updated_at":"2018-11-16T14:56:34Z","closed_at":"2018-11-16T14:56:34Z","author_association":"NONE","body":"Issue: \r\nWe are seeing an issue of memory consumption by a particular node (for queues 1.6 GB out of 1.9 GB total and close to 500K erlang processes being used) in a 3 node cluster while the other's consumption is very less (< 50MB).\r\n\r\nThere's no activity happening in the system currently - hardly one or two queues are getting a 2 messages/sec (size of the message is also ~10 bytes).\r\n\r\nIt's using the policy of min-masters and queues (number) looks properly distributed.\r\n\r\nBackground: 2 days backs identified couple of queues, having no consumers, with more messages (around 300K messages). We have purged the messages and restarted the consumers that solved the issue. After this we have identified the above observation of more Memory usage & Erlang process. Cluster health is good and no other issues identified.\r\n\r\nIs the above scenario causes any problem in not releasing the memory? If yes, how can we recover the same? \r\n\r\nAdditional Details:\r\nRabbiMQ version: 3.6.10\r\n\r\nErlang Version: Erlang/OTP 19 [erts-8.3.5]\r\n\r\nOutput of the Plugins:\r\nConfigured: E = explicitly enabled; e = implicitly enabled\r\n | Status:   * = running on rabbit@centralinfravm1\r\n |/\r\n[e*] amqp_client                       3.6.10\r\n[e*] cowboy                            1.0.4\r\n[e*] cowlib                            1.0.2\r\n[  ] rabbitmq_amqp1_0                  3.6.10\r\n[  ] rabbitmq_auth_backend_ldap        3.6.10\r\n[  ] rabbitmq_auth_mechanism_ssl       3.6.10\r\n[  ] rabbitmq_consistent_hash_exchange 3.6.10\r\n[  ] rabbitmq_event_exchange           3.6.10\r\n[E*] rabbitmq_federation               3.6.10\r\n[E*] rabbitmq_federation_management    3.6.10\r\n[E*] rabbitmq_jms_topic_exchange       3.6.10\r\n[E*] rabbitmq_management               3.6.10\r\n[e*] rabbitmq_management_agent         3.6.10\r\n[  ] rabbitmq_management_visualiser    3.6.10\r\n[  ] rabbitmq_mqtt                     3.6.10\r\n[  ] rabbitmq_recent_history_exchange  3.6.10\r\n[  ] rabbitmq_sharding                 3.6.10\r\n[  ] rabbitmq_shovel                   3.6.10\r\n[  ] rabbitmq_shovel_management        3.6.10\r\n[  ] rabbitmq_stomp                    3.6.10\r\n[  ] rabbitmq_top                      3.6.10\r\n[  ] rabbitmq_tracing                  3.6.10\r\n[  ] rabbitmq_trust_store              3.6.10\r\n[e*] rabbitmq_web_dispatch             3.6.10\r\n[  ] rabbitmq_web_mqtt                 3.6.10\r\n[  ] rabbitmq_web_mqtt_examples        3.6.10\r\n[  ] rabbitmq_web_stomp                3.6.10\r\n[  ] rabbitmq_web_stomp_examples       3.6.10\r\n[  ] sockjs                            0.3.4\r\n\r\nCluster nodes information: \r\n![image](https://user-images.githubusercontent.com/15726141/48621152-ad98bf80-e9c8-11e8-96a7-6f63d2ae32d2.png)\r\n\r\nQueues/Memory from the /queues API:\r\n\r\nNode Name | Number of Queues | Sum of Memory (in Bytes)\r\nrabbit@centralinfravm1 | 180 | 43386992 (~43MB)\r\nrabbit@centralinfravm2 | 177 | 18253984 (~18 MB)\r\nrabbit@centralinfravm3 | 179 | 19671552 (~19 MB)\r\nGrand Total | 536 | 81312528 (~80MB)\r\n\r\nMemory Consumption details of a node:\r\n![image](https://user-images.githubusercontent.com/15726141/48621106-7d512100-e9c8-11e8-8b9d-fac976497080.png)\r\n\r\nOS Details:\r\nLinux centralinfravm1 3.13.0-141-generic # 190-Ubuntu SMP Fri Jan 19 12:52:38 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux\r\n\r\nTried the rest stats DB & performed the garbage collection as well (\r\nrabbitmqctl eval '[garbage_collect(P) || P <- processes()].') but it reduced close to 250MB but didn't release the rest.\r\n\r\n"}}
{"actor":{"display_login":"michaelklishin"}}