title: Extracting a substring after a match position using grok in logstash
body: <p>Objective : I have a log file from where I want to extract the amount details after the string <strong>Amount:::</strong> in the below given log file.</p>
  
  <p>What I have Done so far: Since it is a Custom Parsing, I have created a custom pattern using RegEx and I am trying to Implement it using logstash.</p>
  
  <p>here is my log file -   </p>
  
  <pre><code>28-04-2017 14:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 3000.00  
  28-04-2017 12:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 31000.00  
  28-04-2017 14:15:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 10000.00  
  28-04-2017 11:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 9000.00  
  28-04-2017 08:15:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 7000.00
  </code></pre>
  
  <p>I have used Regex to find the string <strong>Amount:::</strong><br>
  Note : I want to extract the sub string which is coming after the string <strong>Amount:::</strong>  </p>
  
  <p>here are my Custom Patterns I have used in Grok:<br>
  (but it doesn't yield good results)  </p>
  
  <pre><code>CUSTOM_AMOUNT (?&lt;= - Amount::: ).*    
  CUSTOM_AMOUNT (?&lt;=Amount::: )%{BASE16FLOAT}
  </code></pre>
  
  <p>here is my <code>logstacsh.conf</code>-    </p>
  
  <pre><code>input { 
      file {
         path =&gt; "D:\elk\data\amnt_parse.txt"
         type =&gt; "customgrok"
         start_position =&gt; "beginning"
         sincedb_path =&gt; "/dev/null"
           } 
        }  
   filter{ 
         if[type]== "customgrok" {
  
              if "_grokparsefailure" in [tags] { 
                                grok { 
                                     patterns_dir =&gt; "D:\elk\logstash-5.2.1\vendor\bundle\jruby\1.9\gems\logstash-patterns-core-4.0.2\patterns\custom" 
  
                                      match =&gt; { "message" =&gt; "%{CUSTOM_AMOUNT:amount" } 
                                      add_field =&gt; { "subType" =&gt; "Amount"           } 
  
      } 
  
      }
      }  
   mutate {
        gsub =&gt; ['message', "\t", " "] 
          }  } }
  
   output {
       stdout {
           codec =&gt; "rubydebug"
              }
       elasticsearch {
           index =&gt; "amnt_parsing_change"
            hosts =&gt;"localhost"
  
              }
              }   
  </code></pre>
  
  <p>Our intension is to Visualize and to perform aggregation operations based on the extracted substring using Kibana and Elasticsearch.<br>
  but it stores the log file into the variable <code>"message"</code>. as you can see here, <code>match =&gt; { "message" =&gt; "%{CUSTOM_AMOUNT:amount" }</code>.  </p>
  
  <p>here is how the line is stored inside <code>"message"</code>, when I tried to view it in <strong>Kibana</strong> -   </p>
  
  <pre><code>"message": "28-04-2017 11:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 9000.00\r",  
  "message": "28-04-2017 12:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 31000.00\r",    
  "message": "28-04-2017 11:45:50 INFO  abcinfo (ABC_TxnLog_ServiceImpl.java295) - Amount::: 9000.00\r",  
  </code></pre>
  
  <p>Logstash file is loading the Data(log file) and Index is also getting created but <code>Custom Pattern</code> isn't giving expected result. 
  what are possibilities to extract the sub string which I have mentioned above ? or do we have any alternatives?  </p>