<p>I am trying to setup EFK (ElasticSearch 8, FluentD and Kibana) stack on K8S cluster (on-premises)</p>
<p>I followed this <a href="https://phoenixnap.com/kb/elasticsearch-helm-chart" rel="nofollow noreferrer">link</a> to install elasticsearch and installed it using helm charts and  followed this <a href="https://medium.com/kubernetes-tutorials/cluster-level-logging-in-kubernetes-with-fluentd-e59aa2b6093a" rel="nofollow noreferrer">link</a> to install fluentd</p>
<p><strong>Output of fluentd and elasticsearch pods</strong></p>
<pre><code>[root@ctrl01 ~]#  kubectl get pods
NAME                                                     READY   STATUS    RESTARTS   AGE
elasticsearch-master-0                                   1/1     Running   0          136m

[root@ctrl01 ~]#  kubectl get pods -n kube-system
NAME                                                            READY   STATUS    RESTARTS   AGE
fluentd-cnb7p                                                   1/1     Running   0          107m
fluentd-dbxjk                                                   1/1     Running   0          107m
</code></pre>
<p>However, elasticsearch log was piled up with the following warning messages</p>
<pre><code>2021-10-18 12:13:12 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2021-10-18 12:13:42 +0000 error_class=&quot;Elasticsearch::Transport::Transport::Errors::BadRequest&quot; error=&quot;[400] {\&quot;error\&quot;:{\&quot;root_cause\&quot;:[{\&quot;type\&quot;:\&quot;illegal_argument_exception\&quot;,\&quot;reason\&quot;:\&quot;Action/metadata line [1] contains an unknown parameter [_type]\&quot;}],\&quot;type\&quot;:\&quot;illegal_argument_exception\&quot;,\&quot;reason\&quot;:\&quot;Action/metadata line [1] contains an unknown parameter [_type]\&quot;},\&quot;status\&quot;:400}&quot; plugin_id=&quot;out_es&quot;
2021-10-18 12:13:12 +0000 [warn]: suppressed same stacktrace
</code></pre>
<p><strong>Conf file (tailored output)</strong></p>
<pre><code>2021-10-18 12:09:10 +0000 [info]: using configuration file: &lt;ROOT&gt;
  &lt;match fluent.**&gt;
    @type null
  &lt;/match&gt;
  &lt;source&gt;
    @type tail
    @id in_tail_container_logs
    path /var/log/containers/*.log
    pos_file /var/log/fluentd-containers.log.pos
    tag kubernetes.*
    read_from_head true
    format json
    time_format %Y-%m-%dT%H:%M:%S.%NZ
  &lt;/source&gt;
  &lt;source&gt;
    @type tail
    @id in_tail_minion
    path /var/log/salt/minion
    pos_file /var/log/fluentd-salt.pos
    tag salt
    format /^(?&lt;time&gt;[^ ]* [^ ,]*)[^\[]*\[[^\]]*\]\[(?&lt;severity&gt;[^ \]]*) *\] (?&lt;message&gt;.*)$/
    time_format %Y-%m-%d %H:%M:%S
  &lt;/source&gt;
</code></pre>
<p>I am not sure which <code>'type'</code> field it refers to.  I am unable to find an example of ElasticSearch 8 for <code>match</code> and <code>source</code> directives to compare</p>
<p>It seems <code>type</code> field is <strong>not</strong> supported from ES 8 onwards but I am not sure on that.  Kindly let me know the reason for the error</p>

<p>I faced similar errors when I tried to use elasticsearch 8.2.3 with fluentBit 1.9.5. I could see elastic was sending logs but could not see any data in kibana webpage due to which could not create indices and saw the above error in fluent-bit pod logs. I followed <a href="https://github.com/fluent/fluent-bit/issues/5138" rel="noreferrer">this github issue</a> and added <strong>Suppress_Type_Name On</strong> under outputs: section in my fluent-bit helm chart values.yaml file and it worked fine after that.</p>
<pre><code>      [OUTPUT]
          Name  es
          Match *
          Host  {{ .Values.global.backend.es.host }}
          Port  {{ .Values.global.backend.es.port }}
          Logstash_Format Off
          Retry_Limit False
          Type  _doc
          Time_Key @timestamp
          Replace_Dots On
          Suppress_Type_Name On
          Index {{ .Values.global.backend.es.index }}
      {{ .Values.extraEntries.output }}
</code></pre>
