title: Logstash and nested JSON from Monlog; Why arrays are converted to JSON string?
body: <p>I am using PHP with Monolog. I am outputting logs to JSON file and using Gelf to Logstash which then sends logs to ElasticSearch.</p>
  
  <p>The problem I have is that I am missing the <code>extra</code> object in Kibana and the <code>tags</code> field gets interpreted as string instead of nested object.</p>
  
  <p>Any idea how to convince Logstash/Kibana, so the inner JSON field are parsed as fields/objects not as JSON string?</p>
  
  <p>This is how it looks like in Kibana. </p>
  
  <pre><code>{
     "_index":"logstash-2018.08.30",
     "_type":"doc",
     "_id":"TtHbiWUBc7g5w1yM8X6f",
     "_version":1,
     "_score":null,
     "_source":{
        "ctxt_task":"taskName",
        "@version":"1",
        "http_method":"GET",
        "user_agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0",
        "level":6,
        "message":"Finished task",
        "tags":"{\"hostname\":\"28571f0dc7e1\",\"region\":\"eu-west-1\",\"environment\":\"local\",\"processUniqueId\":\"5b87a4d843c20\"}",
        "url":"/assets/Logo.jpg",
        "ctxt_controller":"ControllerName",
        "memory_usage":"4 MB",
        "referrer":"https://local.project.net/account/login",
        "facility":"logger",
        "memory_peak_usage":"4 MB",
        "ctxt_timeElapsed":0.05187487602233887,
        "@timestamp":"2018-08-30T08:03:37.386Z",
        "ip":"172.18.0.1",
        "ctxt_start":1535616217.33417,
        "type":"gelf",
        "host":"18571f0dc7e9",
        "source_host":"172.18.0.8",
        "server":"local.project.net",
        "ctxt_end":1535616217.386045,
        "version":"1.0"
     },
     "fields":{
        "@timestamp":[
           "2018-08-30T08:03:37.386Z"
        ]
     },
     "sort":[
        1535616217386
     ]
  }
  </code></pre>
  
  <p>My log looks like:</p>
  
  <pre><code>{
     "message":"Finished task",
     "context":{
        "controller":"ControllerName",
        "task":"taskName",
        "timeElapsed":0.02964186668395996,
        "start":1535614742.840069,
        "end":1535614742.869711,
        "content":""
     },
     "level":200,
     "level_name":"INFO",
     "channel":"logger",
     "datetime":{
        "date":"2018-08-30 08:39:02.869850",
        "timezone_type":3,
        "timezone":"Europe/London"
     },
     "extra":{
        "memory_usage":"14 MB",
        "memory_peak_usage":"14 MB",
        "tags":{
           "hostname":"28571f0dc7e1",
           "region":"eu-west-1",
           "environment":"local",
           "processUniqueId":"5b879f16be3f1"
        }
     }
  }
  </code></pre>
  
  <p>My logstash conf:</p>
  
  <pre><code>input {
      tcp {
          port =&gt; 5000
      }
      gelf {
          port =&gt; 12201
          type =&gt; gelf
          codec =&gt; "json"
      }
  }
  
  output {
      elasticsearch {
          hosts =&gt; "172.17.0.1:9201"
      }
  }
  </code></pre>
  
  <p>My monolog config:</p>
  
  <pre><code>$gelfTransport = new \Gelf\Transport\UdpTransport(LOG_GELF_HOST, LOG_GELF_PORT);
              $gelfPublisher = new \Gelf\Publisher($gelfTransport);
              $gelfHandler = new \Monolog\Handler\GelfHandler($gelfPublisher, static::$logVerbosity);
              $gelfHandler-&gt;setFormatter(new \Monolog\Formatter\GelfMessageFormatter());
  
              // This is to prevent application from failing if `GelfHandler` fails for some reason
              $ignoreErrorHandlers = new \Monolog\Handler\WhatFailureGroupHandler([
                  $gelfHandler
              ]);
              $logger-&gt;pushHandler($ignoreErrorHandlers);
  </code></pre>
  
  <p>EDIT:
  So far my finding is that this is caused by <code>GelfMessageFormatter</code> converting they arrays to JSON:</p>
  
  <pre><code>$val = is_scalar($val) || null === $val ? $val : $this-&gt;toJson($val);
  </code></pre>
  
  <p>When <code>netcat</code> is used along with the nested JSON, e.g.:</p>
  
  <pre><code>echo -n '{
  "field": 1,
  "nestedField1": {"nf1": 1.1, "nf2": 1.2, "2nestedfield":{"2nf1":1.11, "2nf2":1.12}}
  }' | gzip -c | nc -u -w1 bomcheck-logstash 12201
  </code></pre>
  
  <p>then data in Kibana looks OK </p>