{"action":"opened","issue":{"url":"https://api.github.com/repos/influxdata/influxdb/issues/10178","repository_url":"https://api.github.com/repos/influxdata/influxdb","labels_url":"https://api.github.com/repos/influxdata/influxdb/issues/10178/labels{/name}","comments_url":"https://api.github.com/repos/influxdata/influxdb/issues/10178/comments","events_url":"https://api.github.com/repos/influxdata/influxdb/issues/10178/events","html_url":"https://github.com/influxdata/influxdb/issues/10178","id":349099693,"node_id":"MDU6SXNzdWUzNDkwOTk2OTM=","number":10178,"title":"Memory gradually grows until crash","user":{"login":"philippreston","id":4885250,"node_id":"MDQ6VXNlcjQ4ODUyNTA=","avatar_url":"https://avatars3.githubusercontent.com/u/4885250?v=4","gravatar_id":"","url":"https://api.github.com/users/philippreston","html_url":"https://github.com/philippreston","followers_url":"https://api.github.com/users/philippreston/followers","following_url":"https://api.github.com/users/philippreston/following{/other_user}","gists_url":"https://api.github.com/users/philippreston/gists{/gist_id}","starred_url":"https://api.github.com/users/philippreston/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philippreston/subscriptions","organizations_url":"https://api.github.com/users/philippreston/orgs","repos_url":"https://api.github.com/users/philippreston/repos","events_url":"https://api.github.com/users/philippreston/events{/privacy}","received_events_url":"https://api.github.com/users/philippreston/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-09T12:08:23Z","updated_at":"2018-08-09T12:08:23Z","closed_at":null,"author_association":"NONE","body":"__System info:__  [Include InfluxDB version, operating system name, and other relevant details]\r\n\r\n- InfluxDB 1.6.0 - TSI index\r\n- Running in Docker\r\n- Ubuntu 16.04.2 LTS\r\n- 4.4.0-77-generic x86_64\r\n- 8 CPU / 32G \r\n- KVM, SSD backed storage\r\n\r\n__Steps to reproduce:__\r\n\r\nI originally reported this in #9938.\r\n\r\nWe are seeing continual, gradual memory growth until ultimately the kernel kills influx and the instance is restarted.  Most of the time this appears to run out of memory during compaction, however, I believe that's as a result of the memory has risen to a point where the expected spike will take it into the OOM state.  See the graph below:\r\n\r\n<img width=\"1961\" alt=\"43151693-6e703306-8f64-11e8-86c5-a7b76a72cd90\" src=\"https://user-images.githubusercontent.com/4885250/43897103-ec8d26b6-9bd2-11e8-9763-379abf0e326d.png\">\r\n\r\nAnd to zoom in at the crash:\r\n\r\n<img width=\"1963\" alt=\"43151782-a41cd39c-8f64-11e8-8db0-5d069aa6965a\" src=\"https://user-images.githubusercontent.com/4885250/43897125-003bbf7e-9bd3-11e8-815e-860bb49255c7.png\">\r\n\r\nThe data is written by telegraf from across a sub-set of our platform.  Current cardinality:\r\n\r\n```> show series cardinality\r\ncardinality estimation\r\n----------------------\r\n9873\r\n```\r\n\r\nWe had previously run on 4-core / 16G of RAM and moved up to 8-core 32G which seems in line with a recommendation for our result set. \r\n\r\n__Expected behaviour:__ \r\n\r\nNot expecting the continual memory growth.\r\n\r\n__Actual behaviour:__ \r\n\r\nMemory gradually grows until killed by the kernel with OOM.   The pattern certainly looks a pattern consistent with leaked memory.\r\n\r\n__Additional info:__ \r\n\r\n[profiles.tar.gz](https://github.com/influxdata/influxdb/files/2274065/profiles.tar.gz)\r\n[influxdb.conf.zip](https://github.com/influxdata/influxdb/files/2274075/influxdb.conf.zip)\r\n\r\n(This happens every couple of days so happy to produce anything that is needed)\r\n"}}
{"actor":{"display_login":"philippreston"}}