<p>I have an issue where my beatmetric is caught by my http pipeline.</p>

<p>Both Logstash, Elastic and Metricbeat is running in Kubernetes.</p>

<p>My beatmetric is setup to send to Logstash on port 5044 and log to a file in /tmp. This works fine. But whenever I create a pipeline with an <code>http</code> input, this seems to <em>also</em> catch beatmetric inputs and send them to <code>index2</code> in Elastic as defined in the <code>http</code> pipeline. </p>

<p>Why does it behave like this?</p>

<p><strong>/usr/share/logstash/pipeline/http.conf</strong></p>

<pre><code>input {
  http {
    port =&gt; "8080"
  }
}

output {

  #stdout { codec =&gt; rubydebug }

  elasticsearch {

    hosts =&gt; ["http://my-host.com:9200"]
    index =&gt; "test2"
  }
}
</code></pre>

<p><strong>/usr/share/logstash/pipeline/beats.conf</strong></p>

<pre><code>input {
    beats {
        port =&gt; "5044"
    }
}

output {
    file {
        path =&gt; '/tmp/beats.log'
        codec =&gt; "json"
    }
}
</code></pre>

<p><strong>/usr/share/logstash/config/logstash.yml</strong></p>

<pre><code>pipeline.id: main
pipeline.workers: 1
pipeline.batch.size: 125
pipeline.batch.delay: 50
http.host: "0.0.0.0"
http.port: 9600
config.reload.automatic: true
config.reload.interval: 3s
</code></pre>

<p><strong>/usr/share/logstash/config/pipeline.yml</strong></p>

<pre><code>- pipeline.id: main
  path.config: "/usr/share/logstash/pipeline"
</code></pre>

d00dle
<p>Even if you have multiple config files, they are read as a single pipeline by logstash, concatenating the inputs, filters and outputs, if you need to run then as separate pipelines you have two options.</p>

<p>Change your <code>pipelines.yml</code> and create differents <code>pipeline.id</code>, each one pointing to one of the config files.</p>

<pre><code>- pipeline.id: beats
  path.config: "/usr/share/logstash/pipeline/beats.conf"
- pipeline.id: http
  path.config: "/usr/share/logstash/pipeline/http.conf"
</code></pre>

<p>Or you can use <code>tags</code> in your <code>input</code>, <code>filter</code> and <code>output</code>, for example:</p>

<pre><code>input {
  http {
    port =&gt; "8080"
    tags =&gt; ["http"]
  }
  beats {
    port =&gt; "5044"
    tags =&gt; ["beats"]
  }
}
output {
 if "http" in [tags] {
      elasticsearch {
        hosts =&gt; ["http://my-host.com:9200"]
        index =&gt; "test2"
      }
  }
 if "beats" in [tags] {
      file {
        path =&gt; '/tmp/beats.log'
        codec =&gt; "json"
      }
  }
}

</code></pre>

<p>Using the <code>pipelines.yml</code> file is the recommended way to running <a href="https://www.elastic.co/guide/en/logstash/6.7/multiple-pipelines.html" rel="nofollow noreferrer">multiple pipelines</a></p>
