title: Parse ClamAV logs in Bash script using Regex to insert in MySQL
body: <p>Morning/Evening all,</p>
  
  <p>I've got a problem where I'm making a script for work that uses ClamAV to scan for malware, and then place it's results in MySQL by taking the resultant ClamAV logs using grep with awk to convert the right parts of the log to a variable. The problem I have is that whilst I have done the summary ok, the syntax of detections makes it slightly more difficult. I'm no expert at regex by all means and this is a bit of a learning experience, so there is probably a far better way of doing it than I have!</p>
  
  <p>The lines I'm trying to parse looks like these:</p>
  
  <pre><code>/net/nas/vol0/home/recep/SG4rt.exe: Worm.SomeFool.P FOUND
  /net/nas/vol0/home/recep/SG4rt.exe: moved to '/srv/clamav/quarantine/SG4rt.exe'
  </code></pre>
  
  <p>As far as I was able to establish, I need a positive lookbehind to match what happens after and before the colon, without actually matching the colon or the space after it, and I can't see a clear way of doing it from RegExr without it thinking I'm trying to look for two colons. To make matters worse, we sometimes get these too...</p>
  
  <pre><code>WARNING: Can't open file /net/nas/vol0/home/laser/samples/sample1.avi: Permission denied
  </code></pre>
  
  <p>The end result is that I can build a MySQL query that inserts the path, malware found and where it was moved to or if there was an error then the path, then the error encountered so as to convert each element to a variable contents in a while statement.</p>
  
  <p>I've done the scan summary as follows:</p>
  
  <p>Summary looks like:</p>
  
  <pre><code>----------- SCAN SUMMARY -----------
  Known viruses: 329
  Engine version: 0.97.1
  Scanned directories: 17350
  Scanned files: 50342
  Infected files: 3
  Total errors: 1
  Data scanned: 15551.73 MB
  Data read: 16382.67 MB (ratio 0.95:1)
  Time: 3765.236 sec (62 m 45 s)
  </code></pre>
  
  <p>Parsing like this:</p>
  
  <pre><code>SCANNED_DIRS=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Scanned directories" | awk '{gsub("Scanned directories: ", "");print}')
  SCANNED_FILES=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Scanned files" | awk '{gsub("Scanned files: ", "");print}')
  INFECTED=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Infected files" | awk '{gsub("Infected files: ", "");print}')
  DATA_SCANNED=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Data scanned" | awk '{gsub("Data scanned: ", "");print}')
  DATA_READ=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Data read" | awk '{gsub("Data read: ", "");print}')
  TIME_TAKEN=$(cat /srv/clamav/$IY-scan-$LOGTIME.log | grep "Time" | awk '{gsub("Time: ", "");print}')
  END_TIME=$(date +%s)
  mysql -u scanner_parser --password=removed sc_live -e "INSERT INTO bs.live.bs_jobstat VALUES (NULL, '$CURRTIME', '$PID', '$IY', '$SCANNED_DIRS', '$SCANNED_FILES', '$INFECTED', '$DATA_SCANNED', '$DATA_READ', '$TIME_TAKEN', '$END_TIME');"
  rm -f /srv/clamav/$IY-scan-$LOGTIME.log
  </code></pre>
  
  <p>Some of those variables are from other parts of the script and can be ignored. The reason I'm doing this is to save logfile clutter and have a simple web based overview of the status of the system.</p>
  
  <p>Any clues? Am I going about all this the wrong way? Thanks for help in advance, I do appreciate it!</p>