{"action":"created","issue":{"url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/24","repository_url":"https://api.github.com/repos/cloudfoundry/loggregator","labels_url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/24/comments","events_url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/24/events","html_url":"https://github.com/cloudfoundry/loggregator/issues/24","id":294375782,"number":24,"title":"Adapters under load fills up memory and start swapping","user":{"login":"tyyko","id":4589522,"avatar_url":"https://avatars2.githubusercontent.com/u/4589522?v=4","gravatar_id":"","url":"https://api.github.com/users/tyyko","html_url":"https://github.com/tyyko","followers_url":"https://api.github.com/users/tyyko/followers","following_url":"https://api.github.com/users/tyyko/following{/other_user}","gists_url":"https://api.github.com/users/tyyko/gists{/gist_id}","starred_url":"https://api.github.com/users/tyyko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tyyko/subscriptions","organizations_url":"https://api.github.com/users/tyyko/orgs","repos_url":"https://api.github.com/users/tyyko/repos","events_url":"https://api.github.com/users/tyyko/events{/privacy}","received_events_url":"https://api.github.com/users/tyyko/received_events","type":"User","site_admin":false},"labels":[{"id":725082454,"url":"https://api.github.com/repos/cloudfoundry/loggregator/labels/unscheduled","name":"unscheduled","color":"D9D9D9","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-02-05T12:27:40Z","updated_at":"2018-02-13T07:57:29Z","closed_at":"2018-02-13T07:57:29Z","author_association":"NONE","body":"We have a test system with 1000 deployed applications. Each application is bound to its own syslog drain service.\r\n\r\nStarting with 2 adapter VMs (2 cpu, 4gb ram) we gradually increased the number of logs sent by the test applications and we experienced some memory issues in the adapters.\r\n\r\nIncreasing the load caused an increase of the memory usage up to 100% and made the VM start swapping.\r\n\r\nThe expected behaviour would be to drop incoming logs before starting to swap.\r\n\r\nDoppler are 2 VMs 2cpu, 4gb ram each.\r\n\r\nThis was tested on loggregator-release v99 (the one used for etcd-removal migration) and the 101.4 (the one in cf-deployment 1.9.0) "},"comment":{"url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/comments/365179267","html_url":"https://github.com/cloudfoundry/loggregator/issues/24#issuecomment-365179267","issue_url":"https://api.github.com/repos/cloudfoundry/loggregator/issues/24","id":365179267,"user":{"login":"tyyko","id":4589522,"avatar_url":"https://avatars2.githubusercontent.com/u/4589522?v=4","gravatar_id":"","url":"https://api.github.com/users/tyyko","html_url":"https://github.com/tyyko","followers_url":"https://api.github.com/users/tyyko/followers","following_url":"https://api.github.com/users/tyyko/following{/other_user}","gists_url":"https://api.github.com/users/tyyko/gists{/gist_id}","starred_url":"https://api.github.com/users/tyyko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tyyko/subscriptions","organizations_url":"https://api.github.com/users/tyyko/orgs","repos_url":"https://api.github.com/users/tyyko/repos","events_url":"https://api.github.com/users/tyyko/events{/privacy}","received_events_url":"https://api.github.com/users/tyyko/received_events","type":"User","site_admin":false},"created_at":"2018-02-13T07:57:29Z","updated_at":"2018-02-13T07:57:29Z","author_association":"NONE","body":"Thanks for the info! I would close this in favour of #25 as you mentioned changing the logic under heavy load on both adapter and rlp. \r\n\r\nWould be cool to have this numbers you mentioned about cached envelopes in the docs as well!\r\n\r\nThanks and regards,\r\nMichael"}}
{"actor":{"display_login":"tyyko"}}