<p>I am installing EFK stack to 100 days old cluster. Fluentd will pick up the logs of 100 days and will start sending to Elastic.
Is there a provision that fluentd starts aggregating the log from today and not from beginning life cycle of the cluster?</p>

Arpan Sharma
<p>If you look into <a href="https://docs.fluentd.org/input/tail#limit_recently_modified" rel="nofollow noreferrer">Fluentd documentation</a>, you can find <code>limit_recently_modified</code> flag, which allows limit the watching files that the modification time is within the specified time range.</p>

<p>Here's how the <code>limit_recently_modified</code> can be used in conf file:</p>

<pre><code>...
&lt;source&gt;
    exclude_path ["/var/log/wedge/*/*/MattDaemon*.log"]
    path_key source
    format none
    read_from_head true
    tag foo.*
    path /var/log/wedge/*/*/*.log
    pos_file /var/log/td-agent/wedgelog
    limit_recently_modified 86400s
    @type tail
&lt;/source&gt;
...
</code></pre>

<p>Another option is to use <a href="https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-log.html#filebeat-input-log-ignore-older" rel="nofollow noreferrer">Filebeat</a> (instead of Fluentd), where you can find <code>ignore_older</code> flag. Filebeat ignores any files that were modified before the specified timespan.</p>

<p>I hope it will helps you.</p>
