<p>I am trying to run fluentd as a daemonset on kubernetes cluster (GKE). The config is getting parsed successfully, then the plugins are receiving shutdown signal with few warn messages. There are no error messages. I tried increasing the verbosity level and following is the output of pods:</p>

<pre><code>fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: starting fluentd worker pid=9 ppid=1 worker=0
fluentd-7przp fluentd 2018-09-08 11:02:46 +0000 [info]: #0 fluent/log.rb:322:info: fluentd worker is now running worker=0
fluentd-sr764 fluentd 2018-09-08 11:02:50 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:45.151774166 +0000 record={"worker"=&gt;0, "message"=&gt;"fluentd worker is now running worker=0"}
fluentd-dhwnx fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.029522363 +0000 record={"worker"=&gt;0, "message"=&gt;"fluentd worker is now running worker=0"}
fluentd-7przp fluentd 2018-09-08 11:02:51 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.info" time=2018-09-08 11:02:46.538377182 +0000 record={"worker"=&gt;0, "message"=&gt;"fluentd worker is now running worker=0"}
fluentd-sr764 fluentd 2018-09-08 11:02:55 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:02:50.153922217 +0000 record={"error"=&gt;"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;", "location"=&gt;"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=&gt;"fluent.info", "time"=&gt;2018-09-08 11:02:45.151774166 +0000, "record"=&gt;{"worker"=&gt;0, "message"=&gt;"fluentd worker is now running worker=0"}, "message"=&gt;"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=&gt;0, \"message\"=&gt;\"fluentd worker is now running worker=0\"}"}
fluentd-sr764 fluentd 2018-09-08 11:03:10 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.warn" time=2018-09-08 11:03:05.168427649 +0000 record={"error"=&gt;"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;", "location"=&gt;"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=&gt;"fluent.warn", "time"=&gt;2018-09-08 11:03:00.165843014 +0000, "record"=&gt;{"error"=&gt;"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;", "location"=&gt;"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=&gt;"fluent.warn", "time"=&gt;2018-09-08 11:02:55.156840516 +0000, "record"=&gt;{"error"=&gt;"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;", "location"=&gt;"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=&gt;"fluent.warn", "time"=&gt;2018-09-08 11:02:50.153922217 +0000, "record"=&gt;{"error"=&gt;"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;", "location"=&gt;"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'", "tag"=&gt;"fluent.info", "time"=&gt;2018-09-08 11:02:45.151774166 +0000, "record"=&gt;{"worker"=&gt;0, "message"=&gt;"fluentd worker is now running worker=0"}, "message"=&gt;"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.info\" time=2018-09-08 11:02:45.151774166 +0000 record={\"worker\"=&gt;0, \"message\"=&gt;\"fluentd worker is now running worker=0\"}"}, "message"=&gt;"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:50.153922217 +0000 record={\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.info\", \"time\"=&gt;2018-09-08 11:02:45.151774166 +0000, \"record\"=&gt;{\"worker\"=&gt;0, \"message\"=&gt;\"fluentd worker is now running worker=0\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.info\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}\"}"}, "message"=&gt;"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:02:55.156840516 +0000 record={\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.warn\", \"time\"=&gt;2018-09-08 11:02:50.153922217 +0000, \"record\"=&gt;{\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.info\", \"time\"=&gt;2018-09-08 11:02:45.151774166 +0000, \"record\"=&gt;{\"worker\"=&gt;0, \"message\"=&gt;\"fluentd worker is now running worker=0\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.info\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.warn\\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\\"error\\\"=&gt;\\\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\\\", \\\"location\\\"=&gt;\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\", \\\"tag\\\"=&gt;\\\"fluent.info\\\", \\\"time\\\"=&gt;2018-09-08 11:02:45.151774166 +0000, \\\"record\\\"=&gt;{\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}, \\\"message\\\"=&gt;\\\"dump an error event: error_class=NoMethodError error=\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\" location=\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\" tag=\\\\\\\"fluent.info\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\"worker\\\\\\\"=&gt;0, \\\\\\\"message\\\\\\\"=&gt;\\\\\\\"fluentd worker is now running worker=0\\\\\\\"}\\\"}\"}"}, "message"=&gt;"dump an error event: error_class=NoMethodError error=\"undefined method `[]' for nil:NilClass\" location=\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\" tag=\"fluent.warn\" time=2018-09-08 11:03:00.165843014 +0000 record={\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.warn\", \"time\"=&gt;2018-09-08 11:02:55.156840516 +0000, \"record\"=&gt;{\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.warn\", \"time\"=&gt;2018-09-08 11:02:50.153922217 +0000, \"record\"=&gt;{\"error\"=&gt;\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\", \"location\"=&gt;\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\", \"tag\"=&gt;\"fluent.info\", \"time\"=&gt;2018-09-08 11:02:45.151774166 +0000, \"record\"=&gt;{\"worker\"=&gt;0, \"message\"=&gt;\"fluentd worker is now running worker=0\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.info\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.warn\\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\\"error\\\"=&gt;\\\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\\\", \\\"location\\\"=&gt;\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\", \\\"tag\\\"=&gt;\\\"fluent.info\\\", \\\"time\\\"=&gt;2018-09-08 11:02:45.151774166 +0000, \\\"record\\\"=&gt;{\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}, \\\"message\\\"=&gt;\\\"dump an error event: error_class=NoMethodError error=\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\" location=\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\" tag=\\\\\\\"fluent.info\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\"worker\\\\\\\"=&gt;0, \\\\\\\"message\\\\\\\"=&gt;\\\\\\\"fluentd worker is now running worker=0\\\\\\\"}\\\"}\"}, \"message\"=&gt;\"dump an error event: error_class=NoMethodError error=\\\"undefined method `[]' for nil:NilClass\\\" location=\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\" tag=\\\"fluent.warn\\\" time=2018-09-08 11:02:55.156840516 +0000 record={\\\"error\\\"=&gt;\\\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\\\", \\\"location\\\"=&gt;\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\", \\\"tag\\\"=&gt;\\\"fluent.warn\\\", \\\"time\\\"=&gt;2018-09-08 11:02:50.153922217 +0000, \\\"record\\\"=&gt;{\\\"error\\\"=&gt;\\\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\\\", \\\"location\\\"=&gt;\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\", \\\"tag\\\"=&gt;\\\"fluent.info\\\", \\\"time\\\"=&gt;2018-09-08 11:02:45.151774166 +0000, \\\"record\\\"=&gt;{\\\"worker\\\"=&gt;0, \\\"message\\\"=&gt;\\\"fluentd worker is now running worker=0\\\"}, \\\"message\\\"=&gt;\\\"dump an error event: error_class=NoMethodError error=\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\" location=\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\" tag=\\\\\\\"fluent.info\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\"worker\\\\\\\"=&gt;0, \\\\\\\"message\\\\\\\"=&gt;\\\\\\\"fluentd worker is now running worker=0\\\\\\\"}\\\"}, \\\"message\\\"=&gt;\\\"dump an error event: error_class=NoMethodError error=\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\" location=\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\" tag=\\\\\\\"fluent.warn\\\\\\\" time=2018-09-08 11:02:50.153922217 +0000 record={\\\\\\\"error\\\\\\\"=&gt;\\\\\\\"#&lt;NoMethodError: undefined method `[]' for nil:NilClass&gt;\\\\\\\", \\\\\\\"location\\\\\\\"=&gt;\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\", \\\\\\\"tag\\\\\\\"=&gt;\\\\\\\"fluent.info\\\\\\\", \\\\\\\"time\\\\\\\"=&gt;2018-09-08 11:02:45.151774166 +0000, \\\\\\\"record\\\\\\\"=&gt;{\\\\\\\"worker\\\\\\\"=&gt;0, \\\\\\\"message\\\\\\\"=&gt;\\\\\\\"fluentd worker is now running worker=0\\\\\\\"}, \\\\\\\"message\\\\\\\"=&gt;\\\\\\\"dump an error event: error_class=NoMethodError error=\\\\\\\\\\\\\\\"undefined method `[]' for nil:NilClass\\\\\\\\\\\\\\\" location=\\\\\\\\\\\\\\\"/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'\\\\\\\\\\\\\\\" tag=\\\\\\\\\\\\\\\"fluent.info\\\\\\\\\\\\\\\" time=2018-09-08 11:02:45.151774166 +0000 record={\\\\\\\\\\\\\\\"worker\\\\\\\\\\\\\\\"=&gt;0, \\\\\\\\\\\\\\\"message\\\\\\\\\\\\\\\"=&gt;\\\\\\\\\\\\\\\"fluentd worker is now running worker=0\\\\\\\\\\\\\\\"}\\\\\\\"}\\\"}\"}"}
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [debug]: #0 fluent/log.rb:302:debug: preparing shutdown output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:24 +0000 [info]: #0 fluent/log.rb:322:info: shutting down output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-dhwnx fluentd 2018-09-08 11:03:25 +0000 [warn]: #0 fluent/log.rb:342:warn: dump an error event: error_class=NoMethodError error="undefined method `[]' for nil:NilClass" location="/fluentd/vendor/bundle/ruby/2.3.0/gems/fluent-plugin-elasticsearch-2.10.1/lib/fluent/plugin/out_elasticsearch_dynamic.rb:268:in `eval'" tag="fluent.debug" time=2018-09-08 11:03:24.151685730 +0000 record={"message"=&gt;"fluentd main process get SIGTERM"}
fluentd-sr764 fluentd-dhwnx fluentd-7przp fluentd fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-dhwnxfluentd-sr764  fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
fluentd 2018-09-08 11:03:25 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="kubelet_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="apiserver_out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on output plugin type=:elasticsearch_dynamic plugin_id="out_es"
fluentd-7przp fluentd 2018-09-08 11:03:25 +0000 [debug]: #0 fluent/log.rb:302:debug: calling terminate on filter plugin type=:parser plugin_id="myapp_filter"
fluentd-7przp fluentd 2018-09-08 11:03:26 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished with status 0
</code></pre>

<p>fluentd.conf:</p>

<pre><code>@include systemd.conf
@include kubernetes.conf

# Start of fluent.conf
&lt;filter kubernetes.var.log.containers.myapp-provider**.log&gt;
  @type parser
  @id myapp_filter
  key_name log
  reserve_data true
  remove_key_name_field true
  &lt;parse&gt;
    @type multiline
    format_firstline /^[A-Z]/
    format1 /^(?&lt;level&gt;[^ ]*)[ \t]+\[(?&lt;time&gt;[^\]]*)\] \[(?&lt;thread&gt;[^\]]*)\] \[(?&lt;request&gt;[^\]]*)\] (?&lt;class&gt;[^ ]*): (?&lt;message&gt;.*)$/
    time_format %Y-%m-%d %H:%M:%S,%L %z
  &lt;/parse&gt;
&lt;/filter&gt;

&lt;match kubelet&gt;
   @type elasticsearch_dynamic
   @id kubelet_out_es
   log_level info
   include_tag_key true
   host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
   port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
   scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
   ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
   #user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
   #password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
   reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
   index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
   include_timestamp true
   #logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
   #logstash_format true
   &lt;buffer&gt;
     flush_thread_count 8
     flush_interval 5s
     chunk_limit_size 2M
     queue_limit_length 32
     retry_max_interval 30
     retry_forever true
   &lt;/buffer&gt;
&lt;/match&gt;

&lt;match kube-apiserver&gt;
   @type elasticsearch_dynamic
   @id apiserver_out_es
   log_level info
   include_tag_key true
   host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
   port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
   scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
   ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
   #user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
   #password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
   reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
   index_name fluentd-${tag_parts[0]+ "-" + Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
   include_timestamp true
   #logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
   #logstash_format true
   &lt;buffer&gt;
     flush_thread_count 8
     flush_interval 5s
     chunk_limit_size 2M
     queue_limit_length 32
     retry_max_interval 30
     retry_forever true
   &lt;/buffer&gt;
&lt;/match&gt;

&lt;match **&gt;
   @type elasticsearch_dynamic
   @id out_es
   log_level info
   include_tag_key true
   host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
   port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
   scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
   ssl_verify "#{ENV['FLUENT_ELASTICSEARCH_SSL_VERIFY'] || 'true'}"
   #user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
   #password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
   reload_connections "#{ENV['FLUENT_ELASTICSEARCH_RELOAD_CONNECTIONS'] || 'true'}"
   index_name fluentd-${record['kubernetes']['container_name']}-${Time.at(time).getlocal("+05:30").strftime(@logstash_dateformat)}
   include_timestamp true
   #logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_LOGSTASH_PREFIX'] || 'logstash'}"
   #logstash_format true
   &lt;buffer&gt;
     flush_thread_count 8
     flush_interval 5s
     chunk_limit_size 2M
     queue_limit_length 32
     retry_max_interval 30
     retry_forever true
   &lt;/buffer&gt;
&lt;/match&gt;
</code></pre>

<p>Can anyone please help to debug this. Thanks in advance!</p>

<p>This could be a kubernetes bug as mentioned here - <a href="https://github.com/kubernetes/kubernetes/issues/51376" rel="nofollow noreferrer">https://github.com/kubernetes/kubernetes/issues/51376</a></p>

<p>Although this issue is closed as of now, I would suggest you to try running the same <code>daemonset</code> in <code>default</code> namespace.
Even I tried running it in <code>kube-system</code> namespace but its failing and was running successfully in the <code>default</code> namespace.</p>
