title: Logstash: Using offset as document_id
body: <p>I have following setup:</p>
  
  <blockquote>
    <p>FileBeat -> Logstash -> Elasticsearch -> Kibana (all 5.1.1)</p>
  </blockquote>
  
  <p>When I push the log file (<code>JSON</code>) into Filebeat, and If I try to see it in Kibana Interface, the same logs are added 3-4 time (duplicates).
  After checking the FileBeat logs I came to know that it may be because Filebeat does not receive an acknowledgement for sent logs and so it keeps on resending.
  to stop receiving duplicate documents I think I will have to use a <code>document_id</code> in logstash config file.
  <br>i.e. </p>
  
  <pre><code>output
  { 
      elasticsearch { 
          document_id =&gt; "%{offset}"
          index =&gt; "ap-index"
          hosts =&gt; ["localhost:9222"]
      }
  }
  </code></pre>
  
  <p>My question is, Is the offset field unique for each document?
  and Is that a correct way to stop receiving duplicates?</p>