title: index based on field date in elasticsearch
body: <p>I am importing a syslog using logstash (version 1.5.3) into elasticsearch(version 1.7.1) using the following configuration.</p>
  
  <pre><code>input{
    file {
      path =&gt; "somepath\*.log"
    }
  }
  filter{
      grok {
        match =&gt; { "message" =&gt; "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:message_hostname} %{DATA:message_program}(?:\[%{POSINT:message_pid}\])?: %{GREEDYDATA:user_message}" }
      }
      date {
        match =&gt; [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
  }
  output {
    elasticsearch{
      cluster =&gt; somecluster 
      host =&gt; localhost 
      index =&gt; "logindex-%{+YYYY-MM-dd}"
    }
  }
  </code></pre>
  
  <p>My index is created based on the current date and time i.e. <code>logindex-2015-08-07</code>.</p>
  
  <p>I want to create the index based on the date <code>syslog_timestamp</code> and not the current date using the above format <code>{+YYYY-MM-dd}</code></p>
  
  <p>So If the log had a timestamp <code>2015-01-01</code>, my index should be created as <code>logindex-2015-01-01</code> and not <code>logindex-2015-08-07</code></p>
  
  <p><strong>EDIT</strong></p>
  
  <p>Log Input Used:</p>
  
  <pre><code>Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message
  </code></pre>
  
  <p><strong>Logstash Debug Output</strong></p>
  
  <pre><code>←[36mfilter received {:event=&gt;{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]:(root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT"}, :level=&gt;:debug, :file=&gt;"(eval)", :line=&gt;"69", :method=&gt;"filter_func"}
   ←[0m
   ←[36mRunning grok filter {:event=&gt;#&lt;LogStash::Event:0x166d6250 @metadata_accessors=#&lt;LogStash::Util::Accessors:0x14fe0ed7 @store={}, @lut={}&gt;, @cancelled=false, @data={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT"}, @metadata={}, @accessors=#&lt;LogStash::Util::Accessors:0x6ee8ffaf @store={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT"}, @lut={"host"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root)
  CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT"}, "host"]}&gt;&gt;, :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb", :line=&gt;"283", :method=&gt;"filter"}
  ←[0m
  ←[36mRegexp match object {:names=&gt;["SYSLOGTIMESTAMP:message_timestamp", "SYSLOGHOST:message_hostname", "DATA:message_program", "POSINT:message_pid", "GREEDYDATA:user_message"], :captures=&gt;["Jul 27 07:49:01", "Server1", "CRON", "21009", "(root) CMD LTest Message\r"], :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.2/lib/grok-pure.rb", :line=&gt;"179", :method=&gt;"match_and_capture"}
  ←[0m
  ←[36mfilters/LogStash::Filters::Grok: adding value to field {:field=&gt;"received_at", :value=&gt;["%{@timestamp}"], :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.3-java/lib/logstash/util/decorators.rb", :line=&gt;"28", :method=&gt;"add_fields"}
  
  ←[0m
  ←[36mEvent now:  {:event=&gt;#&lt;LogStash::Event:0x166d6250 @metadata_accessors=#&lt;LogStash::Util::Accessors:0x14fe0ed7 @store={}, @lut={}&gt;, @cancelled=false, @data={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, @metadata={}, @accessors=#&lt;LogStash::Util::Accessors:0x6ee8ffaf @store={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, @lut={"host"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "host"], "message"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009","user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message"], "message_timestamp"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_timestamp"], "message_hostname"=&gt;[{"message"=&gt;"Jul27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_hostname"], "message_program"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTestMessage\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_program"], "message_pid"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_pid"], "user_message"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "user_message"], "@timestamp"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "@timestamp"], "received_at"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01","message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "received_at"]}&gt;&gt;, :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-1.0.0/lib/logstash/filters/grok.rb", :line=&gt;"303", :method=&gt;"filter"}
  
  ←[0m
  ←[36mDate filter: received event {:type=&gt;nil, :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/logstash-filter
  -date-1.0.0/lib/logstash/filters/date.rb", :line=&gt;"206", :method=&gt;"filter"}
  ←[0m←[36mDate filter looking for field {:type=&gt;nil, :field=&gt;"syslog_timestamp", :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/
  jruby/1.9/gems/logstash-filter-date-1.0.0/lib/logstash/filters/date.rb", :line=&gt;
  "209", :method=&gt;"filter"}
  ←[0m
  ←[36moutput received {:event=&gt;{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]:(root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, :level=&gt;:debug, :file=&gt;"(eval)", :line=&gt;"76", :method=&gt;"output_func"}
  ←[0m
  ←[36mFlushing output {:outgoing_count=&gt;1, :time_since_last_flush=&gt;22.048, :outgoing_events=&gt;{nil=&gt;[["index", {:_id=&gt;nil, :_index=&gt;"%index-2015-08-14", :_type=&gt;"logs", :_routing=&gt;nil},
  #&lt;LogStash::Event:0x166d6250 
  @metadata_accessors=#&lt;LogStash::Util::Accessors:0x14fe0ed7 @store={"retry_count"=&gt;0}, @lut={}&gt;, @cancelled=false, @data={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, @metadata={"retry_count"=&gt;0}, 
  @accessors=#&lt;LogStash::Util::Accessors:0x6ee8ffaf @store={"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, @lut={"host"=&gt;[{"message"=&gt;sage\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, @lut={"host"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r","received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "host"], "message"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message"], "message_timestamp"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_timestamp"], "message_hostname"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_hostname"], "message_program"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_program"], "message_pid"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "message_pid"], "user_message"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "user_message"], "@timestamp"=&gt;[{"message"=&gt;"Jul 2707:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "@timestamp"], "received_at"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "received_at"], "type"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r","received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "type"], "syslog_timestamp"=&gt;[{"message"=&gt;"Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r", "@version"=&gt;"1", "@timestamp"=&gt;"2015-08-14T07:34:53.215Z", "host"=&gt;"HOST-LT", "message_timestamp"=&gt;"Jul 27 07:49:01", "message_hostname"=&gt;"Server1", "message_program"=&gt;"CRON", "message_pid"=&gt;"21009", "user_message"=&gt;"(root) CMD LTest Message\r", "received_at"=&gt;"2015-08-14T07:34:53.215Z"}, "syslog_timestamp"]}&gt;&gt;]]}, :batch_timeout=&gt;1, :force=&gt;nil, :final=&gt;nil, :level=&gt;:debug, :file=&gt;"/Folder/logstash-1.5.3/logstash-1.5.3/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/buffer.rb", :line=&gt;"207", :method=&gt;"buffer_flush"}
  
  ←[0m{              "message" =&gt; "Jul 27 07:49:01 Server1 CRON[21009]: (root) CMD LTest Message\r",
               "@version" =&gt; "1",
             "@timestamp" =&gt; "2015-08-14T07:34:53.215Z",
                   "host" =&gt; "HOST-LT",
      "message_timestamp" =&gt; "Jul 27 07:49:01",
       "message_hostname" =&gt; "Server1",
        "message_program" =&gt; "CRON",
            "message_pid" =&gt; "21009",
           "user_message" =&gt; "(root) CMD LTest Message\r",
            "received_at" =&gt; "2015-08-14T07:34:53.215Z"
  }
  </code></pre>