<p>I really hope somebody would help me out with this question, as I have been trying to figure it out for days.</p>

<p>I have container running in kubernetes in GKE. In /var/log/containers/my_container.log, I have something like this (among some other logs with different formats):</p>

<pre><code>{"log":"17-Oct-2017;04:36:29.744 : [main] [server:] [id:] [yt:] ERROR no.myproject.service.Server - call failed for some reason\n","stream":"stdout","time":"2017-10-17T04:36:29.750702216Z"}
</code></pre>

<p>This log appears on Stackdriver (Fluentd output in GKE) as INFO log and like:</p>

<pre><code>23:02:32.000 17-Oct-2017;04:36:29.744 : [main] [server:] [id:] [yt:] ERROR no.myproject.service.Server - call failed for some reason
</code></pre>

<p>So</p>

<pre><code>23:02:32.000 
</code></pre>

<p>is added to it (which is the normal behavior of Stackdriver). I will refer to this format as format 2. </p>

<p>As this log message actually is an ERROR log message (based on its payload content) I want it to appear as ERROR in Stackdriver (Fluentd).</p>

<p>I am trying:</p>

<pre><code>&lt;filter reform.**&gt;
  type parser
  format /^(?&lt;time&gt;\d{2} [^\s]*) : (?&lt;message2&gt;[^ \]]*)\] (?&lt;message3&gt;[^ \]]*)\] (?&lt;message4&gt;[^ \]]*)\] (?&lt;message5&gt;[^ \]]*)\] (?&lt;severity&gt;\w)\s+(?&lt;log2222&gt;.*)/
  reserve_data true
  suppress_parse_error_log false
  key_name log
&lt;/filter&gt;
</code></pre>

<p>hoping to get the severity of the message change to ERROR and also getting the content of the [..] fields in the log as some new key/values (in this case message2: main, etc).</p>

<p>But after adding this filter to my config file, the output logs are still as before and I don't see any change.</p>

<p>What am I missing? When I am writting my Regex pattern, I am not sure if I actually should consider the "log" field of the message in the log file in kubernetes or the one that I called format 2 (with the time added to it - on Stackdriver).</p>

<p>I would really appreciate any advice, it would be a great help.</p>

<p>I don't know the tool you're using, but it looks like first part of your regex does not match the exact format of the prefixed time in your log string.</p>

<p>Actually <code>\d{2}</code> will only match 2 digits.</p>

<p>To match the entire time prefix you may use <code>(?:\d{2}:){2}\d{2}\.\d{3}</code> instead.</p>

<p>One additional point regarding severity: you wrote <code>(?&lt;severity&gt;\w)</code> which captures only one word character. You may use <code>(?&lt;severity&gt;\w+)</code> to match several characters.</p>

<p>Your regex would then become:</p>

<pre><code>^(?&lt;time&gt;(?:\d{2}:){2}\d{2}\.\d{3} [^\s]*) : (?&lt;message2&gt;[^ \]]*)\] (?&lt;message3&gt;[^ \]]*)\] (?&lt;message4&gt;[^ \]]*)\] (?&lt;message5&gt;[^ \]]*)\] (?&lt;severity&gt;\w+)\s+(?&lt;log2222&gt;.*)
</code></pre>

<p>That <a href="https://regex101.com/r/o5lu2j/1" rel="nofollow noreferrer">demo</a> shoes a match.</p>
