title: Regexp for parse log with fluentd
body: <p>I'm trying to parse application log with some regexp. I was able to parse the timestamp. But after that, If I try to add more expressions to the fluentd format the first attribute "time" disappears with giving me an exception. And no records are matched.</p>
  
  <p>I'm using Fluentular.</p>
  
  <p>Log message will be something like:</p>
  
  <pre><code>Date=[2018-04-11 08:44:30,219] Thread=[20] Level=[INFO] EventId=[2] Message=[Request finished in 1.1825ms 200 text/plain; charset=utf-8]
  </code></pre>
  
  <p>This is the first regexp with the first key (There is an issue with milliseconds but is not important). In the attributes I can see the key Time with this value.</p>
  
  <p><a href="http://fluentular.herokuapp.com/parse?regexp=%28%3F%3Ctime%3E%28%3F%3C%3D%5C%5B%29.%2B%3F%28%3F%3D%5C%5D%29%29&amp;input=Date%3D%5B2018-04-11%2008%3A44%3A30%2C219%5D%20Thread%3D%5B20%5D%20Level%3D%5BINFO%5D%20EventId%3D%5B2%5D%20Message%3D%5BRequest%20finished%20in%201.1825ms%20200%20text%2Fplain%3B%20charset%3Dutf-8%5D&amp;time_format=%25Y-%25m-%25d%20%25H%3A%25M%3A%25S" rel="nofollow noreferrer">First regexp ok</a></p>
  
  <p>then If I try to isolate more info on the log I lose the first key and I'm unable to parse any other data.</p>
  
  <p><a href="http://fluentular.herokuapp.com/parse?regexp=%28%3F%3Ctime%3E%28%3F%3C%3D%5C%5B%29.%2B%3F%28%3F%3D%5C%5D%29%29%20%28%3F%3Cmethod%3E%5B%5E%20%5D*%29&amp;input=Date%3D%5B2018-04-11%2008%3A44%3A30%2C219%5D%20Thread%3D%5B20%5D%20Level%3D%5BINFO%5D%20EventId%3D%5B2%5D%20Message%3D%5BRequest%20finished%20in%201.1825ms%20200%20text%2Fplain%3B%20charset%3Dutf-8%5D&amp;time_format=%25Y-%25m-%25d%20%25H%3A%25M%3A%25S" rel="nofollow noreferrer">Regexp with second key</a></p>
  
  <p>What I'm doing wrong?</p>
  
  <p>Thanks</p>
  
  <p>UPDATE:</p>
  
  <p>based on comments now I have this expression</p>
  
  <pre><code>(?&lt;time&gt;\[(?&lt;time&gt;[^\]\[]+)])\s+(?&lt;Thread&gt;\S+)\s+(?&lt;Level&gt;\S+)\s+(?&lt;EventId&gt;\S+)\s+(?&lt;Message&gt;[^ ].*$)
  </code></pre>
  
  <p><a href="http://fluentular.herokuapp.com/parse?regexp=%28%3F%3Ctime%3E%5C%5B%28%3F%3Ctime%3E%5B%5E%5C%5D%5C%5B%5D%2B%29%5D%29%5Cs%2B%28%3F%3CThread%3E%5CS%2B%29%5Cs%2B%28%3F%3CLevel%3E%5CS%2B%29%5Cs%2B%28%3F%3CEventId%3E%5CS%2B%29%5Cs%2B%28%3F%3CMessage%3E%5B%5E%20%5D.*%24%29&amp;input=Date%3D%5B2018-04-11%2008%3A44%3A30%2C219%5D%20Thread%3D%5B20%5D%20Level%3D%5BINFO%5D%20EventId%3D%5B2%5D%20Message%3D%5BRequest%20finished%20in%201.1825ms%20200%20text%2Fplain%3B%20charset%3Dutf-8%5D&amp;time_format=%25Y-%25m-%25d%20%25H%3A%25M%3A%25S" rel="nofollow noreferrer">Regexp 3</a></p>
  
  <p>Is almost perfect. The only problem is that, the expression also get the square bracket</p>
  
  <p>Key   Thread
  Value Thread=[20]</p>
  
  <p>I don't know if this expression can be improved to avoid the</p>
  
  <p>"Thread=[]"</p>
  
  <p>UPDATE 2:</p>
  
  <p>In order to do test, I installed fluentd via apt with the plugin and do some tests and my final setup is something like</p>
  
  <pre><code>&lt;parse&gt;
          @type kv
          time_key Date
          types Date:time:%Y-%m-%d %H:%M:%S,Thread:integer,Level:string,EventId:integer,Message:string
          kv_delimiter /\]\s+/
          kv_char "=["
  </code></pre>
  
  <p>Seems better now:</p>
  
  <pre><code>2018-04-11 08:44:30.219000000 +0200 kv_log: {"Thread":20,"Level":"INFO","EventId":2,"Message":"Request finished in 1.1825ms 200 text/plain; charset=utf-8]"}
  </code></pre>