<p>I installed a filebeat -> logstash -> elasticsearch -> kibana stack in Kubernetes with helm charts :</p>

<pre><code>helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator
helm install --name elastic --namespace monitoring incubator/elasticsearch --set client.replicas=1,master.replicas=2,data.replicas=1

helm install --name logstash --namespace monitoring incubator/logstash -f logstash_values.yaml

helm install --name filebeat stable/filebeat -f filebeat_values.yaml

helm install stable/kibana --name kibana --namespace monitoring 
</code></pre>

<p>The logs are indexed in ES, but the "message" contains the whole string, not the defined fields. My grok filter doesn't seem to work in logstash conf.</p>

<p>The is no documentation on <a href="https://github.com/helm/charts/tree/master/incubator/logstash" rel="nofollow noreferrer">https://github.com/helm/charts/tree/master/incubator/logstash</a> about how to set the patterns.</p>

<p>Here is what I tried :</p>

<p>my log's format :</p>

<pre><code>10-09-2018 11:57:55.906 [Debug] [LOG] serviceName - Technical - my specific message - correlationId - userId - data - operation - error - stackTrace escaped on one line
</code></pre>

<p>logstash_values.yaml (from <a href="https://github.com/helm/charts/blob/master/incubator/logstash/values.yaml" rel="nofollow noreferrer">https://github.com/helm/charts/blob/master/incubator/logstash/values.yaml</a>) :</p>

<pre><code>elasticsearch:
  host: elasticsearch-client.default.svc.cluster.local
  port: 9200

patterns:
   main: |-
     (?&lt;time&gt;(?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)\.(?:[0-9]){3})} [(?&lt;logLevel&gt;.*)] [(?&lt;code&gt;.*)] (?&lt;caller&gt;.*) - (?&lt;logMessageType&gt;.*) - (?&lt;message&gt;.*) - (?&lt;correlationId&gt;.*) - (?&lt;userId&gt;.*) - (?&lt;data&gt;.*) - (?&lt;operation&gt;.*) - (?&lt;error&gt;.*) - (?&lt;stackTrace&gt;.*)

inputs:
  main: |-
    input {
      beats {
        port =&gt; 5044
      }
    }

filters:

outputs:
  main: |-
    output {
      elasticsearch {
        hosts =&gt; ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"]
        manage_template =&gt; false
        index =&gt; "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type =&gt; "%{[@metadata][type]}"
      }
    }
</code></pre>

<p>This becomes a Kubernetes configMap "logstash-patterns" :</p>

<pre><code>apiVersion: v1
kind: ConfigMap
data:
  main: (?&lt;time&gt;(?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)\.(?:[0-9]){3}) [(?&lt;code&gt;.*)] [(?&lt;logLevel&gt;.*)] (?&lt;service&gt;.*) - (?&lt;logMessageType&gt;.*) - (?&lt;message&gt;.*) - (?&lt;correlationId&gt;.*) - (?&lt;userId&gt;.*) - (?&lt;data&gt;.*) - (?&lt;operation&gt;.*) - (?&lt;error&gt;.*) - (?&lt;stackTrace&gt;.*)
</code></pre>

<p>I don't see any error logs in logstash pod.</p>

<p>Do you have any idea how to configure patterns in logstash in Kubernetes ?</p>

<p>Thanks.</p>

<p>I was mistaking "pattern" and "filter".</p>

<p>In the Helm chart, "pattern" is for specifying our custom grok patterns (<a href="https://grokdebug.herokuapp.com/patterns" rel="nofollow noreferrer">https://grokdebug.herokuapp.com/patterns</a>) : </p>

<blockquote>
  <p>MY_CUSTOM_ALL_CHARS .*</p>
</blockquote>

<p>My grok filter should be in the filter section :</p>

<pre><code>patterns:
  # nothing here for me 

filters:
  main: |-
    filter {
      grok {
        match =&gt; { "message" =&gt; "\{%{TIMESTAMP_ISO8601:time}\} \[%{DATA:logLevel}\] \[%{DATA:code}\] %{DATA:caller} &amp;\$ %{DATA:logMessageType} &amp;\$ %{DATA:message} &amp;\$ %{DATA:correlationId} &amp;\$ %{DATA:userId} &amp;\$ %{DATA:data} &amp;\$ %{DATA:operation} &amp;\$ %{DATA:error} &amp;\$ (?&lt;stackTrace&gt;.*)" }
        overwrite =&gt; [ "message" ]
      }
      date {
        match =&gt; ["time", "ISO8601"]
        target =&gt; "time"
      }
    }
</code></pre>
