title: Reading from a streaming syslog server using R
body: <p>I am getting CISCO ASA VPN logs in my local live syslog server. Since the data is largely unstructured, I want to process each log and then store the data in a table(preferably hive). </p>
  
  <p>Any code I wrote till now is very basic readLines and connection commands such as opening the syslog and writing the last few lines in an output using-</p>
  
  <pre><code>zz &lt;- file("/home/akashb/write.data","w")
  logread &lt;- file("/var/log/messages","r")
  x &lt;- readLines(logread,-1)
  cat(x[(length(x)-1):length(x)],file=zz,sep="\n")
  </code></pre>
  
  <p>I want to make it in such a way that R takes each new log generated and  process those logs and store the output in some place. That is the connection must be in running always in the background.</p>
  
  <p>Also since the volume of the log will be very high in little time, what kind of pressure I should expect on my RAM, or can it be done in such a way where only one line is processed at a time.</p>
  
  <p>Any help will be very much appreciated.</p>