title: collecting kafka.consumer lag mbean
body: <p>I'm new to kafka and am setting up an ELK stack to collect JMX metrics and log files for analysis and reporting. </p>
  
  <p>Currently, I've setup my logstash to collect kafka mbeans in the following patterns:</p>
  
  <pre><code>    {
              "object_name" : "kafka.*:type=*,name=*,topic=*,partition=*"
      },
      {
              "object_name" : "kafka.*:type=*,name=*,topic=*"
      },
      {
              "object_name" : "kafka.*:type=*,name=*"
      },
      {
              "object_name" : "kafka.*:type=*,client-id=*"
      },
      {
              "object_name" : "kafka.*:type=*,name=*,clientId=*,brokerHost=*,brokerPort=*"
      },
      {
              "object_name" : "kafka.*:type=*,name=*,clientId=*"
      },
      {
              "object_name" : "kafka.*:type=*,broker-id=*,fetcher-id=*"
      },
      {
              "object_name" : "kafka.*:type=*,client-id=*"
      }
  </code></pre>
  
  <p>The issue that I'm having is that I'm missing all <code>kafka.consumer</code> metrics and when I log into to jconsole on the RMI port, I don't see <code>kafka.consumer</code> mbeans there either.</p>
  
  <p>I'm new to this whole JMX/mbeans stuff and kafka, so am I looking in the right place for <code>kafka.consumer</code> metrics? I currently do have active producers and consumers hitting the cluster. </p>