title: read line by line with awk and parse variables
body: <p>I have a script that read log files and parse the data to insert them to mysql table..<BR></p>
  
  <p>My script looks like</p>
  
  <pre><code>while read x;do
  var=$(echo ${x}|cut -d+ -f1) 
  var2=$(echo ${x}|cut -d_ -f3)
  ...
  echo "$var,$var2,.." &gt;&gt; mysql.infile 
  done&lt;logfile
  </code></pre>
  
  <p>The Problem is that log files are thousands of lines and taking hours....</p>
  
  <p>I read that <code>awk</code> is better, I tried, but don't know the syntax to parse the variables...</p>
  
  <p><strong>EDIT:</strong>
  inputs are structure firewall logs so they are pretty large files like </p>
  
  <blockquote>
    <p>@timestamp $HOST reason="idle Timeout" source-address="x.x.x.x"
    source-port="19219" destination-address="x.x.x.x"
    destination-port="53" service-name="dns-udp" application="DNS"....</p>
  </blockquote>
  
  <p>So I'm using a lot of <code>grep</code> for ~60 variables e.g</p>
  
  <pre><code>sourceaddress=$(echo ${x}|grep -P -o '.{0,0} 
  source-address=\".{0,50}'|cut -d\" -f2)
  </code></pre>
  
  <p>if you think <code>perl</code> will be better I'm open to suggestions and maybe a hint how to script it...</p>