{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/beats/issues/3023","repository_url":"https://api.github.com/repos/elastic/beats","labels_url":"https://api.github.com/repos/elastic/beats/issues/3023/labels{/name}","comments_url":"https://api.github.com/repos/elastic/beats/issues/3023/comments","events_url":"https://api.github.com/repos/elastic/beats/issues/3023/events","html_url":"https://github.com/elastic/beats/issues/3023","id":189985919,"number":3023,"title":"filebeat 5.0 startup panic","user":{"login":"jzoe","id":11612990,"avatar_url":"https://avatars.githubusercontent.com/u/11612990?v=3","gravatar_id":"","url":"https://api.github.com/users/jzoe","html_url":"https://github.com/jzoe","followers_url":"https://api.github.com/users/jzoe/followers","following_url":"https://api.github.com/users/jzoe/following{/other_user}","gists_url":"https://api.github.com/users/jzoe/gists{/gist_id}","starred_url":"https://api.github.com/users/jzoe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jzoe/subscriptions","organizations_url":"https://api.github.com/users/jzoe/orgs","repos_url":"https://api.github.com/users/jzoe/repos","events_url":"https://api.github.com/users/jzoe/events{/privacy}","received_events_url":"https://api.github.com/users/jzoe/received_events","type":"User","site_admin":false},"labels":[{"id":78874872,"url":"https://api.github.com/repos/elastic/beats/labels/bug","name":"bug","color":"fc2929","default":true},{"id":294278645,"url":"https://api.github.com/repos/elastic/beats/labels/Filebeat","name":"Filebeat","color":"008000","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-11-17T09:17:31Z","updated_at":"2016-11-20T06:58:42Z","closed_at":null,"body":"Version: filebeat 5.0 + logstash 5.0 + elasticsearch 5.0\r\nSystem: centos 7 (filesystem； XFS)\r\n\r\nHere is my filebeat.yml\r\n```\r\n  1 filebeat.prospectors:\r\n  2 - input_type: log\r\n  3   paths:\r\n  4     - /home/dba/mysqlnode/nodedata1/mysqlslow.log\r\n  5   document_type: mysql_slow_log\r\n  6   multiline.negate: true\r\n  7   multiline.match: after\r\n  8 filebeat.registry_file: /home/dba/tmp/registry\r\n  9 output.logstash:\r\n 10   hosts: [\"localhost:5044\"]\r\n```\r\nAnd my logstash configure like this:\r\n```\r\n  1 input {\r\n  2     beats{\r\n  3         host => \"localhost\"\r\n  4         port => 5045\r\n  5         tags => [\"metric\"]\r\n  6     }\r\n  7     beats{\r\n  8         host => \"localhost\"\r\n  9         port => 5044\r\n 10         codec => multiline {\r\n 11            pattern => \"^# User@Host:\"\r\n 12            negate => true\r\n 13            what => previous\r\n 14         }\r\n 15         tags => [\"mysql_slow_log\"]\r\n 16     }\r\n 17 }\r\n```\r\n\r\nI'm trying to startup filebeat, but I get the panic error as following:\r\n```\r\n[dba@localhost filebeat-5.0.0-linux-x86_64]$ ./filebeat -e -c mysql-slow-log-filebeat.yml -d \"publish\"\r\n  16/11/17 08:49:51.351959 beat.go:264: INFO Home path: [/home/dba/elk/filebeat-5.0.0-linux-x86_64] Config path: [/home/dba/elk/filebeat-5.0.0-linux-x86_64] Data p▽th: [/home/dba/elk/filebeat-5.0.0-linux-x86_64/data] Logs path: [/home/dba/elk/filebeat-5.0.0-linux-x86_64/logs]\r\n2016/11/17 08:49:51.352090 beat.go:174: INFO Setup Beat: filebeat; Version: 5.0.0\r\n2016/11/17 08:49:51.352177 logstash.go:90: INFO Max Retries set to: 3\r\n2016/11/17 08:49:51.352247 outputs.go:106: INFO Activated logstash as output plugin.\r\n2016/11/17 08:49:51.352280 publish.go:234: DBG  Create output worker\r\n2016/11/17 08:49:51.352330 publish.go:276: DBG  No output is defined to store the topology. The server fields might not be filled.\r\n2016/11/17 08:49:51.352375 publish.go:291: INFO Publisher name: localhost.localdomain\r\n2016/11/17 08:49:51.352554 async.go:63: INFO Flush Interval set to: 1s\r\n2016/11/17 08:49:51.352587 async.go:64: INFO Max Bulk Size set to: 2048\r\n2016/11/17 08:49:51.352610 async.go:72: DBG  create bulk processing worker (interval=1s, bulk size=2048)\r\n2016/11/17 08:49:51.352693 beat.go:204: INFO filebeat start running.\r\n2016/11/17 08:49:51.352705 registrar.go:66: INFO Registry file set to: /home/dba/elk/filebeat-5.0.0-linux-x86_64/data/registry\r\n2016/11/17 08:49:51.352721 registrar.go:88: INFO No registry file found under: /home/dba/elk/filebeat-5.0.0-linux-x86_64/data/registry. Creating a new registry file.\r\n2016/11/17 08:49:51.352729 crawler.go:34: INFO Loading Prospectors: 1\r\n2016/11/17 08:49:51.352755 prospector_log.go:40: INFO Load previous states from registry into memory\r\n2016/11/17 08:49:51.352764 prospector_log.go:67: INFO Previous states loaded: 0\r\n2016/11/17 08:49:51.352810 crawler.go:46: INFO Loading Prospectors completed. Number of prospectors: 1\r\n2016/11/17 08:49:51.352815 crawler.go:61: INFO All prospectors are initialised and running with 0 states to persist\r\n2016/11/17 08:49:51.352839 prospector.go:106: INFO Starting prospector of type: log\r\n2016/11/17 08:49:51.352965 log.go:84: INFO Harvester started for file: /home/dba/mysqlnode/nodedata1/mysqlslow.log\r\n2016/11/17 08:49:51.353156 logp.go:219: INFO Metrics logging every 30s\r\n2016/11/17 08:49:51.353221 sync.go:41: INFO Start sending events to output\r\n2016/11/17 08:49:51.353262 registrar.go:211: INFO Starting Registrar\r\npanic: runtime error: invalid memory address or nil pointer dereference\r\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5d55b6]\r\n\r\ngoroutine 29 [running]:\r\npanic(0x8db060, 0xc420012090)\r\n        /usr/local/go/src/runtime/panic.go:500 +0x1a1\r\nregexp.(*Regexp).get(0x0, 0x2)\r\n        /usr/local/go/src/regexp/regexp.go:208 +0x26\r\nregexp.(*Regexp).doExecute(0x0, 0x0, 0x0, 0xc42004dc70, 0x46, 0x47, 0x0, 0x0, 0x0, 0x0, ...)\r\n        /usr/local/go/src/regexp/exec.go:416 +0x40\r\nregexp.(*Regexp).Match(0x0, 0xc42004dc70, 0x46, 0x47, 0x46)\r\n        /usr/local/go/src/regexp/regexp.go:421 +0x8c\r\ngithub.com/elastic/beats/filebeat/harvester/reader.genPatternMatcher.func1(0xc420172d80, 0x58, 0x59, 0xc42004dc70, 0x46, 0x47, 0x1000000010000)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/multiline.go:306 +0xa3\r\ngithub.com/elastic/beats/filebeat/harvester/reader.negatedMatcher.func1(0xc420172d80, 0x58, 0x59, 0xc42004dc70, 0x46, 0x47, 0x47)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/multiline.go:296 +0x61\r\ngithub.com/elastic/beats/filebeat/harvester/reader.(*Multiline).readNext(0xc4200d6180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/multiline.go:189 +0x8b7\r\ngithub.com/elastic/beats/filebeat/harvester/reader.(*Multiline).readFirst(0xc4200d6180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc4201a8468, ...)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/multiline.go:132 +0x167\r\ngithub.com/elastic/beats/filebeat/harvester/reader.(*Multiline).Next(0xc4200d6180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc420040cb0, ...)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/multiline.go:108 +0x62\r\ngithub.com/elastic/beats/filebeat/harvester/reader.(*Limit).Next(0xc4201a06c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/reader/limit.go:17 +0x66\r\ngithub.com/elastic/beats/filebeat/harvester.(*Harvester).Harvest(0xc420147ba0, 0xbe0180, 0xc4201a06c0)\r\n        /go/src/github.com/elastic/beats/filebeat/harvester/log.go:93 +0x24d\r\ngithub.com/elastic/beats/filebeat/prospector.(*Prospector).startHarvester.func1(0xc420105ba0, 0xc420147ba0, 0xbe0180, 0xc4201a06c0)\r\n        /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:223 +0x65\r\ncreated by github.com/elastic/beats/filebeat/prospector.(*Prospector).startHarvester\r\n        /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:224 +0x2ee\r\n```\r\n\r\nIs there any one can help me to figure out what's wrong with my filebeat ?"},"comment":{"url":"https://api.github.com/repos/elastic/beats/issues/comments/261762662","html_url":"https://github.com/elastic/beats/issues/3023#issuecomment-261762662","issue_url":"https://api.github.com/repos/elastic/beats/issues/3023","id":261762662,"user":{"login":"jzoe","id":11612990,"avatar_url":"https://avatars.githubusercontent.com/u/11612990?v=3","gravatar_id":"","url":"https://api.github.com/users/jzoe","html_url":"https://github.com/jzoe","followers_url":"https://api.github.com/users/jzoe/followers","following_url":"https://api.github.com/users/jzoe/following{/other_user}","gists_url":"https://api.github.com/users/jzoe/gists{/gist_id}","starred_url":"https://api.github.com/users/jzoe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jzoe/subscriptions","organizations_url":"https://api.github.com/users/jzoe/orgs","repos_url":"https://api.github.com/users/jzoe/repos","events_url":"https://api.github.com/users/jzoe/events{/privacy}","received_events_url":"https://api.github.com/users/jzoe/received_events","type":"User","site_admin":false},"created_at":"2016-11-20T06:58:42Z","updated_at":"2016-11-20T06:58:42Z","body":"@andrewkroh I specify my multiline pattern in filter section of logstash.cnf  file, which write as:\r\n\r\n```\r\n 19 filter {\r\n 20     if \"mysql_slow_log\" in [tags] {\r\n 21         grok {\r\n 22             match => [ \"message\", \"(?m)^#\\s+User@Host:\\s+%{USER:user}\\[[^\\]]+\\]\\s+@\\s+%{USER:clienthost}\\s+\\[(?:%{IP:clientip})?\\]\\s+Id:\\s+%{NUMBER:id:int}\\n#\\\r\n    s+Schema:\\s+%{USER:schema}\\s+Last_errno:\\s+%{NUMBER:lasterrorno:int}\\s+Killed:\\s+%{NUMBER:killedno:int}\\n#\\s+Query_time:\\s+%{NUMBER:query_time:float}\\s+Lock_ti\r\n    me:\\s+%{NUMBER:lock_time:float}\\s+Rows_sent:\\s+%{NUMBER:rows_sent:int}\\s+Rows_examined:\\s+%{NUMBER:rows_examined:int}\\s+Rows_affected:\\s+%{NUMBER:rows_affected\r\n    :int}\\n#\\s+Bytes_sent:\\s+%{NUMBER:bytes_sent:int}\\n\\s*(?:use\\s+%{USER:usedatabase};\\s*\\n)?SET\\s+timestamp=%{NUMBER:timestamp};\\n\\s*(?<query>(?<action>\\w+)\\b.*)\r\n    \\s*(?:\\n#\\s+Time)?.*$\"]\r\n 23         }\r\n 24         date {\r\n 25             match => [ \"timestamp\", \"UNIX\" ]\r\n 26             #remove_field => [ \"timestamp\" ]\r\n 27         }\r\n 28 \r\n 29     }\r\n 30 \r\n 31 }\r\n```\r\n\r\nShould I put those multiline pattern in the filebeat config file?"}}
{"actor":{"display_login":"jzoe"}}