title: Duplicate data in kibana from logstash
body: <p>I'm trying display some Mongo data that I've been collecting using logstash using the Mongostat tool. It displays things with a suffix like &quot;b&quot;, &quot;k&quot;, &quot;g&quot; to signify byte, kilobyte, gigabyte, which is fine if I'm just reading the output, but I want to throw this into kibana and display it in a graphical format to see trends.</p>
  <p>I've done this with several other log files and everything is fine. When I use a grok filter everything is fine but I've added a Ruby filter and now data seems to be duplicated in all fields other than the logstash generated fields and my new field created in my Ruby filter.</p>
  <p>Here is the relevant parts of my conf file:</p>
  <pre><code>input {
    file {
      path =&gt; &quot;/var/log/mongodb/mongostat.log&quot;
      type =&gt; &quot;mongostat&quot;
      start_position =&gt; &quot;end&quot;
    }
  }
  
  filter {
    if [type] == &quot;mongostat&quot; {
       grok {
         patterns_dir =&gt; &quot;/opt/logstash/patterns&quot;
         match =&gt; [&quot;message&quot;,&quot;###a bunch of filtering that i know works###&quot;]
         add_tag =&gt; &quot;mongostat&quot;
       }
      
          if [mongoMappedQualifier] == 'b' {
              ruby {
                 code =&gt; &quot;event['mongoMappedKB'] = event['mongoMapped'].to_f / 1024&quot;
              }
          }
          if [mongoMappedQualifier] == 'k' {
              ruby {
                 code =&gt; &quot;event['mongoMappedKB'] = event['mongoMapped'].to_f * 1&quot;
              }
          }
          if [mongoMappedQualifier] == 'm' {
              ruby {
                 code =&gt; &quot;event['mongoMappedKB'] = event['mongoMapped'].to_f * 1024&quot;
              }
          }
          if [mongoMappedQualifier] == 'g' {
              ruby {
                 code =&gt; &quot;event['mongoMappedKB'] = event['mongoMapped'].to_f * 1048576&quot;
              }
          }
    }
  }
  
  output {
     if [type] == &quot;mongostat&quot; {
      redis {
         host =&gt; &quot;redis&quot;
         data_type =&gt; &quot;list&quot;
         key =&gt; &quot;logstash-mongostat&quot;
       }
     }
  }
  </code></pre>
  <p>Any idea why or how this can be fixed?</p>
  <p><img src="https://i.stack.imgur.com/htNgI.png" alt="Kibana graph with duplicated data" /></p>