title: Logstash will not process logs that span multiple lines
body: <p>I am trying to parse some local log file, I am running the ELK Stack on my windows machine. here is an example of the logs I am trying to parse.</p>
  
  <pre><code>2015-12-10 13:50:25,487 [http-nio-8080-exec-26] INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- This Message is OK
  2015-12-10 13:50:26,487 [http-nio-8080-exec-26] INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- Journe
  y road update: &lt;rows&gt;
       &lt;row adi="D" date="2015-12-10" garage="TOP"&gt;
            &lt;codeNum order="1"&gt;TP&lt;/codeNum&gt;
            &lt;number order="1"&gt;1001&lt;/number&gt;
            &lt;journeystatus code="RT"&gt;OnRoute&lt;/journeystatus&gt;
       &lt;/row&gt;
  &lt;/rows&gt;
  </code></pre>
  
  <p>the first message works fine through the filters but the second message get split into mutiple messages with <code>_grokparsefailure</code> in the tags part.</p>
  
  <p><strong>Logstash Config File</strong></p>
  
  <pre><code>input {
      file {
          path =&gt; "C:/data/sampleLogs/temp.log"
          type =&gt; "testlog"
          start_position =&gt; "beginning"
      }
  }
  
  filter {
      grok {
          # Parse timestamp data. We need the "(?m)" so that grok (Oniguruma internally) correctly parses multi-line events
          match =&gt; [ "message", [
              "(?m)%{TIMESTAMP_ISO8601:logTimestamp}[ ;]\[%{DATA:threadId}\][ ;]%{LOGLEVEL:logLevel}[ ;]+%{JAVACLASS:JavaClass}[ ;]%{SYSLOG5424SD:TransactionID}[ ;]*%{GREEDYDATA:LogMessage}",
              "(?m)%{TIMESTAMP_ISO8601:logTimestamp}[ ;]\[%{DATA:threadId}\][ ;]%{LOGLEVEL:logLevel}[ ;]+%{JAVAFILE:JavaClass}[ ;]%{SYSLOG5424SD:TransactionID}[ ;]*%{GREEDYDATA:LogMessage}"
              ]
          ]
      }
      # The timestamp may have commas instead of dots. Convert so as to store everything in the same way
      mutate {
          gsub =&gt; [
              # replace all commas with dots
              "logTimestamp", ",", "."
              ]
      }
  
      mutate {
          gsub =&gt; [
              # make the logTimestamp sortable. With a space, it is not! This does not work that well, in the end
              # but somehow apparently makes things easier for the date filter
              "logTimestamp", " ", ";"
              ]
      }
  
      date {
          locale =&gt; "en"
          timezone =&gt; "UTC"
          match =&gt; [ "logTimestamp", "YYYY-MM-dd;HH:mm:ss.SSS" ]
          target =&gt; "@timestamp"
      }
  
      mutate {
          add_field =&gt; { "debug-timestamp" =&gt; "timestampMatched"}
      }
  }
  
  output {
      stdout {
          codec =&gt; rubydebug
      }   
  }
  </code></pre>
  
  <p>When I run </p>
  
  <pre><code>bin\logstash agent -f \ELK-Stack\logstash\conf\01_input.conf
  </code></pre>
  
  <p>in the CMD prompt what is returned is as followed </p>
  
  <pre><code>io/console not supported; tty will not be manipulated
  Default settings used: Filter workers: 4
  Logstash startup completed
  {
              "message" =&gt; "     &lt;row adi=\"D\" date=\"2015-12-10\" garage=\"TOP\"
  &gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.268Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "          &lt;codeNum order=\"1\"&gt;TP&lt;/codeNum&gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.268Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "          &lt;number order=\"1\"&gt;1001&lt;/number&gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.268Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "          &lt;journeystatus code=\"RT\"&gt;OnRoute&lt;/journeys
  tatus&gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.278Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "     &lt;/row&gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.278Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "y road update: &lt;rows&gt;\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-11T12:49:34.268Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
                 "tags" =&gt; [
          [0] "_grokparsefailure"
      ],
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "2015-12-10 13:50:25,487 [http-nio-8080-exec-26] INFO
  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- Journe\r",
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-10T13:50:25.487Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
         "logTimestamp" =&gt; "2015-12-10;13:50:25.487",
             "threadId" =&gt; "http-nio-8080-exec-26",
             "logLevel" =&gt; "INFO",
            "JavaClass" =&gt; "a.b.c.v1.myTestClass",
        "TransactionID" =&gt; "[abcde-1234-12345-b425-12ad]",
           "LogMessage" =&gt; "- Journe\r",
      "debug-timestamp" =&gt; "timestampMatched"
  }
  {
              "message" =&gt; "&lt;/rows&gt;2015-12-10 13:50:25,487 [http-nio-8080-exec-26]
   INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- This Message is OK\r",
  
             "@version" =&gt; "1",
           "@timestamp" =&gt; "2015-12-10T13:50:25.487Z",
                 "host" =&gt; "GMAN",
                 "path" =&gt; "C:/data/sampleLogs/temp.log",
                 "type" =&gt; "testlog",
         "logTimestamp" =&gt; "2015-12-10;13:50:25.487",
             "threadId" =&gt; "http-nio-8080-exec-26",
             "logLevel" =&gt; "INFO",
            "JavaClass" =&gt; "a.b.c.v1.myTestClass",
        "TransactionID" =&gt; "[abcde-1234-12345-b425-12ad]",
           "LogMessage" =&gt; "- This Message is OK\r",
      "debug-timestamp" =&gt; "timestampMatched"
  }
  </code></pre>
  
  <p>I did add mulitline to the top of my filter but it didnt work just gave the following errors and just after the grok. </p>
  
  <pre><code>multiline {
          pattern =&gt; "^201*-**-**- **:**:"
          what =&gt; "previous"
          negate=&gt; true
      }
  </code></pre>
  
  <p>but this did not help just keeps giving me an error message like</p>
  
  <pre><code>Error: Cannot use more than 1 filter worker because the following plugins don't
  work with more than one worker: multiline
  You may be interested in the '--configtest' flag which you can
  use to validate logstash's configuration before you choose
  to restart a running system.
  </code></pre>
  
  <p>So I try running the <code>--configtest</code> as suggested and a new error message appears</p>
  
  <pre><code>Error: Cannot use more than 1 filter worker because the following plugins don't
  work with more than one worker: multiline
  </code></pre>
  
  <p>Can anyone help me solve this and get logstash to process the multiple lines.</p>
  
  <p>Your help is greatly appreciated </p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>As @Alain Collins suggested to using codec with <a href="https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html" rel="nofollow">multiline</a>, here is what the input of my config looks like.</p>
  
  <pre><code>input {
      file {
          path =&gt; "C:/data/sampleLogs/mulline.log"
          codec =&gt; multiline {
              # Grok pattern names are valid! :)
              pattern =&gt; "^%{TIMESTAMP_ISO8601} "
              negate =&gt; true
              what =&gt; previous
          }
          type =&gt; "testlog"
          start_position =&gt; "beginning"
      }
  }
  </code></pre>
  
  <p>G </p>