title: How to import Linux logs into ELK stack for log analysis?
body: <p>I'm building a log analysis environment with the purpose of analyzing linux logs such as: /var/log/auth.log, /var/log/cron, /var/log/syslog, etc. The goal is to be able to upload such a log file and analyze it properly with Kibana/Elasticsearch. To do so, I created a .conf file as seen below, which includes the proper patterns to pars auth.log and the information needed in the input and output section. Unfortunately, when connecting to Kibana I cannot see any data in the &quot;Discover&quot; panel and cannot find the related &quot;index pattern&quot;. I tested the grokk pattern and they works well.</p>
  <pre><code>input {
    file {
      type =&gt; &quot;linux-auth&quot;
      path =&gt; [ &quot;/home/ubuntu/logs/auth.log&quot;]
    }
  filter {
    if [type] == &quot;linux-auth&quot; {
        grok {
          match =&gt; { &quot;message&quot; =&gt; &quot;%{TIMESTAMP_ISO8601:time} %{WORD:method}\[%{POSINT:auth_pid}\]\: %{DATA:message} for %{DATA:user} from %{IPORHOST:IP_address} port %{POSINT:port}&quot; }
        }
        grok {
          match =&gt; { &quot;message&quot; =&gt; &quot;%{TIMESTAMP_ISO8601:time} %{WORD:method}\[%{POSINT:auth_pid}\]\:%{DATA:message} for %{GREEDYDATA:username}&quot; }
        }
    }
  }
  output{
      elasticsearch {
          hosts =&gt; &quot;elasticsearch:9200&quot;
      }
  }
  </code></pre>
  <p>Example of auth.log:</p>
  <pre><code>2018-12-02T14:01:00Z sshd[0000001]: Accepted keyboard-interactive/pam for root from 185.118.167.241 port 64965 ssh2
  2018-12-02T14:02:00Z sshd[0000002]: Failed keyboard-interactive/pam for invalid user ubuntu from 36.104.140.175 port 57512 ssh2
  2018-12-02T14:03:00Z sshd[0000003]: pam_unix(sshd:session): session closed for user root
  </code></pre>