title: Program to generate sample log to feed to logstash?
body: <p>I have written a small java program which generates some dummy logs (writes stuff to a txt file basically). Now I want to feed this data to the ELK stack. Basically logstash should read this data from the txt file and I want to visualize these changes on kibana, just to get a feel of it. </p>
  
  <p>What I basically want to do is then change the speed at which my program writes the dummy logs to the txt file so that I can see the changes on kibana.</p>
  
  <p>I have just started exploring the ELK stack and this might be a completely wrong way to do this kind of analysis. Please do suggest if there are other better ways to do this (considering I don't have actual logs to work with right now)</p>
  
  <p>Edit : @Val</p>
  
  <pre><code>input {
      generator {
          message =&gt; “’83.149.9.216 - - [17/May/2015:10:05:03 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36””
          count =&gt; 10
      }
  }
  </code></pre>
  
  <p>So here is  my <code>logstash.conf</code>:</p>
  
  <pre><code>input {
  
   stdin { }
  
  }
  
  
  filter {
    grok {
      match =&gt; {
        "message" =&gt; '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}'
      }
    }
  
    date {
      match =&gt; [ "timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
      locale =&gt; en
    }
  
    geoip {
      source =&gt; "clientip"
    }
  
    useragent {
      source =&gt; "agent"
      target =&gt; "useragent"
    }
  }
  
  output {
    stdout {
  codec =&gt; plain {
                          charset =&gt; "ISO-8859-1"
                  }
  
  }
    elasticsearch {
      hosts =&gt; "http://localhost:9200"
      index =&gt; "apache_elk_example"
      template =&gt; "./apache_template.json"
      template_name =&gt; "apache_elk_example"
      template_overwrite =&gt; true
    }
  }
  </code></pre>
  
  <p>Now after starting elasticsearch and kabana I do:</p>
  
  <pre><code>cat apache_logs | /usr/local/opt/logstash/bin/logstash -f apache_logs
  </code></pre>
  
  <p>where <code>apache_logs</code> is been fed from my java program:</p>
  
  <pre><code>public static void main(String[] args) {
      // TODO Auto-generated method stub
      try {
          PrintStream out = new PrintStream(new FileOutputStream("/Users/username/Desktop/user/apache_logs"));
          System.setOut(out);
      } catch (FileNotFoundException ex) {
          System.out.print("Exception");
      }
      while(true)
      //for(int i=0;i&lt;5;++i)
      {
          System.out.println(generateRandomIPs() + //other log stuff);
          try {
              Thread.sleep(1000);                 //1000 milliseconds is one second.
          } catch(InterruptedException ex) {
              Thread.currentThread().interrupt();
          }
      }
  }
  </code></pre>
  
  <p>So here is the problem : </p>
  
  <p>Kibana doesn't show me real time visualization i.e. as and when my java program feeds data into the <code>apache_log</code> file it does not show it to me. It only shows only until whatever data was already written into 'apache_log' at the time of execution of :</p>
  
  <pre><code>cat apache_logs | /usr/local/opt/logstash/bin/logstash -f apache_logs
  </code></pre>