{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/logstash/issues/4333","labels_url":"https://api.github.com/repos/elastic/logstash/issues/4333/labels{/name}","comments_url":"https://api.github.com/repos/elastic/logstash/issues/4333/comments","events_url":"https://api.github.com/repos/elastic/logstash/issues/4333/events","html_url":"https://github.com/elastic/logstash/issues/4333","id":121505341,"number":4333,"title":"Logstash 2.1.1 immediate OOM when running simple lumberjack input (regression)","user":{"login":"jamesblackburn","id":55716,"avatar_url":"https://avatars.githubusercontent.com/u/55716?v=3","gravatar_id":"","url":"https://api.github.com/users/jamesblackburn","html_url":"https://github.com/jamesblackburn","followers_url":"https://api.github.com/users/jamesblackburn/followers","following_url":"https://api.github.com/users/jamesblackburn/following{/other_user}","gists_url":"https://api.github.com/users/jamesblackburn/gists{/gist_id}","starred_url":"https://api.github.com/users/jamesblackburn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamesblackburn/subscriptions","organizations_url":"https://api.github.com/users/jamesblackburn/orgs","repos_url":"https://api.github.com/users/jamesblackburn/repos","events_url":"https://api.github.com/users/jamesblackburn/events{/privacy}","received_events_url":"https://api.github.com/users/jamesblackburn/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/logstash/labels/crashes","name":"crashes","color":"e11d21"},{"url":"https://api.github.com/repos/elastic/logstash/labels/needs%20details","name":"needs details","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2015-12-10T15:00:53Z","updated_at":"2016-01-12T08:54:36Z","closed_at":null,"body":"Trying to upgrade from 1.5.5 to 2.1.1 for a simple Logstash `lumberjack -> file` pipeline, I'm observing an immediate OOM on startup.  The OOM appears to be in `org.jruby.ext.zlib.JZlibInflate`\r\n\r\n## Running\r\n`logstash -f \"/ahlinfra/monit/conf/logs.dev.man.com/init.d/../etc/fwd-receiver/*.conf\"  -w1`\r\n\r\n```\r\n$ ps aux|grep 2.1.1\r\nlogstash 36294  255  0.7 5537560 123456 pts/1  Sl+  14:43   0:10 /apps/research/tools/java/current//bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/tmp/heapdump.hprof -Xmx1g -Xss2048k -Djffi.boot.library.path=/apps/research/tools/logstash/2.1.1/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/tmp/heapdump.hprof -XX:HeapDumpPath=/apps/research/tools/logstash/2.1.1/heapdump.hprof -Xbootclasspath/a:/apps/research/tools/logstash/2.1.1/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/apps/research/tools/logstash/2.1.1/vendor/jruby -Djruby.lib=/apps/research/tools/logstash/2.1.1/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /apps/research/tools/logstash/2.1.1/lib/bootstrap/environment.rb logstash/runner.rb agent -f /ahlinfra/monit/conf/logs.dev.man.com/init.d/../etc/fwd-receiver/*.conf -w1\r\n```\r\n## Configuration\r\n\r\n```\r\n$ cat /ahlinfra/monit/conf/logs.dev.man.com/init.d/../etc/fwd-receiver/*.conf\r\ninput {\r\n    lumberjack {\r\n       port => 5043\r\n       ssl_certificate => \"/ahlinfra/monit/bin/../conf/logs.dev.man.com/../common/logs.dev.man.com.crt\"\r\n       ssl_key => \"/ahlinfra/monit/bin/../conf/logs.dev.man.com/../common/logs.dev.man.com.key\"\r\n       congestion_threshold => 30\r\n    }\r\n}\r\nfilter {\r\n    # Strip the base-path prefixes from the filenames\r\n    mutate { \r\n        gsub => [\r\n            \"file\", \"^/var/log/\", \"\",\r\n            \"file\", \"^/data/logs/AHL/\", \"\",\r\n            \"file\", \"^/data/logs/\", \"\",\r\n            \"file\", \"^/local/mongo/logs/\", \"\"\r\n            ]\r\n    }\r\n\r\n    # If there are no slashes left on name after strip above\r\n    # insert the hostname before the filename\r\n    mutate { add_field => { \"[@metadata][svc_slash]\" => \"%{service}/\" } }\r\n    if [@metadata][svc_slash] not in [file] {\r\n        mutate { replace => { \"file\" => \"%{service}/%{file}\" } }\r\n    }\r\n}\r\noutput {\r\n    # End of NFS service tracking\r\n    file {\r\n        path => \"/data/logs/AHL/%{file}-%{+YYYY-MM-dd}\"\r\n        message_format => \"%{message}\"\r\n        flush_interval => 60\r\n    }\r\n}\r\n```\r\n\r\n## Output\r\n\r\n```\r\nYou are using a deprecated config setting \"message_format\" set in file. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. You can achieve the same behavior with the 'line' codec If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>\"message_format\", :plugin=><LogStash::Outputs::File path=>\"/data/logs/AHL/%{file}-%{+YYYY-MM-dd}\", message_format=>\"%{message}\", flush_interval=>60>, :level=>:warn}\r\nSettings: User set filter workers: 1, Default filter workers: 12\r\nLogstash startup completed\r\njava.lang.OutOfMemoryError: Java heap space\r\nDumping heap to /apps/research/tools/logstash/2.1.1/heapdump.hprof ...\r\nHeap dump file created [556095097 bytes in 4.058 secs]\r\n^CSIGINT received. Shutting down the pipeline. {:level=>:warn}\r\nLogstash shutdown completed\r\n```\r\n\r\n## HProf analysis\r\n\r\n![image](https://cloud.githubusercontent.com/assets/55716/11718598/add74a2e-9f4e-11e5-8217-62569ce1a10a.png)\r\n\r\n![image](https://cloud.githubusercontent.com/assets/55716/11718614/c2da7aea-9f4e-11e5-805b-67389217261b.png)\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/logstash/issues/comments/170842119","html_url":"https://github.com/elastic/logstash/issues/4333#issuecomment-170842119","issue_url":"https://api.github.com/repos/elastic/logstash/issues/4333","id":170842119,"user":{"login":"jamesblackburn","id":55716,"avatar_url":"https://avatars.githubusercontent.com/u/55716?v=3","gravatar_id":"","url":"https://api.github.com/users/jamesblackburn","html_url":"https://github.com/jamesblackburn","followers_url":"https://api.github.com/users/jamesblackburn/followers","following_url":"https://api.github.com/users/jamesblackburn/following{/other_user}","gists_url":"https://api.github.com/users/jamesblackburn/gists{/gist_id}","starred_url":"https://api.github.com/users/jamesblackburn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamesblackburn/subscriptions","organizations_url":"https://api.github.com/users/jamesblackburn/orgs","repos_url":"https://api.github.com/users/jamesblackburn/repos","events_url":"https://api.github.com/users/jamesblackburn/events{/privacy}","received_events_url":"https://api.github.com/users/jamesblackburn/received_events","type":"User","site_admin":false},"created_at":"2016-01-12T08:54:36Z","updated_at":"2016-01-12T08:54:36Z","body":"Just to add @suyograo I believe repro. details have been provided?"}}