title: Communication between two fluentd instances using http plugin fails with bad request
body: <p>I have a problem with communication between fluentds using http_out and http_in plugins. I have two kubernetes clusters and I want the fluentd-A in the first cluster to send it's logs to the fluentd-B in the second cluster. But when fluentd-A tries to send the logs it recieves:</p>
  
  <pre><code>#0 got unrecoverable error in primary and no secondary error_class=Fluent::UnrecoverableError error="400 Bad Request 400 Bad Request\n'json' or 'msgpack' parameter is required\n"
  #0 bad chunk is moved to /tmp/fluent/backup/worker0/object_3ff8a73edae8/5a71a08ca19b1b343c8dce1b74c9a963.log
  </code></pre>
  
  <p>When I cat this file the logs in it has the following pattern:</p>
  
  <pre><code>{log: ...., ... other json fields... }
  {log: ...., ... other json fields... }
  {log: ...., ... other json fields... }
  </code></pre>
  
  <p>I think the problem is that the concatenation of json objects is not a json and that is why fluentd-B sends as response 400 Bad Request.</p>
  
  <p>I have tested the fluentd-B with curl with the following requests and it works ok:</p>
  
  <pre><code>curl -X POST -d '{"foo":"bar"}' -H 'Content-Type: application/json' http://&lt;fluentd-B-ip&gt;:9880/app.log
  curl -X POST -d 'json=[{"foo":"bar"},{"abc":"def"},{"xyz":"123"}]' http://&lt;fluentd-B-ip&gt;:9880/app.log
  </code></pre>
  
  <p>Could someone tell me how to pack the logs in the fluentd-A into a valid json? (It's not necessary to be json)</p>
  
  <p>This is the config of the fluentd-A:</p>
  
  <pre><code>&lt;match **&gt;
    @type http
  
    endpoint http://&lt;Fluentd-B-IP&gt;:9880/api.log
    open_timeout 2
    &lt;format&gt;
      @type json
    &lt;/format&gt;
  &lt;/match&gt;
  
  &lt;match fluent.**&gt;
    @type null
  &lt;/match&gt;
  
  &lt;source&gt;
    @type tail
    @id in_tail_container_logs
    path /var/log/containers/*.log
    pos_file /var/log/fluentd-containers.log.pos
    tag "#{ENV['FLUENT_CONTAINER_TAIL_TAG'] || 'kubernetes.*'}"
    exclude_path "#{ENV['FLUENT_CONTAINER_TAIL_EXCLUDE_PATH'] || use_default}"
    read_from_head true
    &lt;parse&gt;
      @type "#{ENV['FLUENT_CONTAINER_TAIL_PARSER_TYPE'] || 'json'}"
      time_format %Y-%m-%dT%H:%M:%S.%NZ
    &lt;/parse&gt;
  &lt;/source&gt;
  
  &lt;filter kubernetes.**&gt;
    @type kubernetes_metadata
    @id filter_kube_metadata
    kubernetes_url "#{ENV['FLUENT_FILTER_KUBERNETES_URL'] || 'https://' + ENV.fetch('KUBERNETES_SERVICE_HOST') + ':' + ENV.fetch('KUBERNETES_SERVICE_PORT') + '/api'}"
    verify_ssl "#{ENV['KUBERNETES_VERIFY_SSL'] || true}"
    ca_file "#{ENV['KUBERNETES_CA_FILE']}"
  &lt;/filter&gt;
  
  &lt;filter kubernetes.**&gt;
    @type grep
    &lt;regexp&gt;
      key $.kubernetes.namespace_name
      pattern /test/
    &lt;/regexp&gt;
  &lt;/filter&gt;
  </code></pre>
  
  <p>This is the config of the fluentd-B:</p>
  
  <pre><code>&lt;source&gt;
    @type http
    port 9880
    bind 0.0.0.0
    body_size_limit 32m
    keepalive_timeout 10s
    &lt;parse&gt;
      @type json
    &lt;/parse&gt;
  
  &lt;/source&gt;
  </code></pre>
  
  <p>I hope someone could help with this, I though that such basic setup should be easy but maybe I missed something important...</p>