title: Including additional measurements with Telegraf plugin inputs.logparser using "grok" patterns (Or regex)
body: <p>I am using telegraf plugin[[inputs.logparser]] to grab the access_log data from Apache based on a local web page I have got running.</p>
  
  <p>Using <code>["%{COMBINED_LOG_FORMAT}"]</code> patterns, I am able to retrieve the default measurements provided by the access_logs, including http_version, request, resp_bytes etc. </p>
  
  <p>I have appended the "Log Format" within httpd.conf file to include the additional "Response time" to each request access_log records with <code>%D</code> at the end, this has been successful when i look at the access_log after implementing.</p>
  
  <p>However I am so far unable to successfully tell Telegraf to acknowledge this new measurement with the inputs.logparser - I am using a grafana dashboard with InfluxDB to monitor this data and it has not yet appeared as an additional measurement.</p>
  
  <p>So far I have attempted the following:</p>
  
  <p>First [[inputs.logparser]] section remains the same throughout my attempts and is always present/active, this seems right in order to be able to obtain the default measurements? </p>
  
  <pre><code>######## default logparser using COMBINED to obtain default access_log measurements ######
  # Stream and parse log file(s).
  [[inputs.logparser]]
    files = ["/var/log/httpd/access_log"]
    from_beginning = true
  
    ## Parse logstash-style "grok" patterns:
    [inputs.logparser.grok]
  
      patterns = ["%{COMBINED_LOG_FORMAT}"
      measurement = "apache_access_log"
      custom_patterns = '''
      '''
  
  </code></pre>
  
  <p>Attempt 1 at matching the response time appended to access_log:</p>
  
  <pre><code>############# Grok/RegEx for matching response time ######################
  # Stream and parse log file(s).
  [[inputs.logparser]]
    ## Log files to parse.
    files = ["/var/log/httpd/access_log"]
    from_beginning = true
  
    ## Parse logstash-style "grok" patterns:
    [inputs.logparser.grok]
      patterns = ["%{METRICS_INCLUDE_RESPONSE}"]
  
      measurement = "apache_access_log"
      custom_patterns = '''
      METRICS_INCLUDE_RESPONSE [%{NUMBER:resp}]
      '''
  </code></pre>
  
  <p>And my 2nd attempt I thought to try normal regular expressions </p>
  
  <pre><code>############# Grok/RegEx for matching response time ######################
  # Stream and parse log file(s).
  [[inputs.logparser]]
    ## Log files to parse.
    files = ["/var/log/httpd/access_log"]
    from_beginning = true
  
    ## Parse logstash-style "grok" patterns:
    [inputs.logparser.grok]
      patterns = ["%{METRICS_INCLUDE_RESPONSE}"]   
      measurement = "apache_access_log"
      custom_patterns = '''
      METRICS_INCLUDE_RESPONSE [%([0-9]{1,3})]
      '''
  </code></pre>
  
  <p>After both of these attempts, the default measurements are still recorded and grabbed fine by Telegraf, but the response time does not appear as an additional measurement. </p>
  
  <p>I believe the issue to be syntax within my custom grok pattern, and that it is not matching as I have intended it to because I am not telling it to pull the correct information? But I am unsure.</p>
  
  <p>I have provided an example of the access_log output below, ALL details are pulled from Telegraf without issue under COMBINED_LOG_FORMAT, except for the number at the end, which is representative of the response time. </p>
  
  <pre><code>10.30.20.32 - - [09/Jan/2020:11:08:14 +0000] "POST /404.php HTTP/1.1" 200 252 "http://10.30.10.77/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36" 600
  10.30.20.32 - - [09/Jan/2020:11:08:15 +0000] "POST /boop.html HTTP/1.1" 200 76 "http://10.30.10.77/404.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36" 472
  </code></pre>