title: Search for multiple strings in many text files, count hits on combinations
body: <p>I'm struggling to automate a reporting exercise, and would appreciate some pointers or advice please.</p>
  
  <p>I have several hundred thousand small (&lt;5kb) text files. Each contains a few variables, and I need to count the number of files that match each combination of variables.</p>
  
  <p>Each file contains a device number, such as /001/ /002/.../006/. </p>
  
  <p>Each file also contains a date string, such as 01.10.14 (dd.mm.yy)</p>
  
  <p>Some files contain a 'status' string which is always "Not Settled"</p>
  
  <p>I need a way to trawl through each file on a Linux system (spread across several subdirectories), and produce a report file that counts 'per device' how many files include each date stamp (6 month range) and for each of those dates, how many contain the status string.</p>
  
  <p>The report might look like this:
  device, date, total count of files
  device, date, total "not settled" count</p>
  
  <p>e.g.</p>
  
  <pre><code>/001/, 01.12.14, 356
  /001/, 01.12.14, 12
  /001/, 02.12.14, 209
  /001/, 02.12.14, 8 
  /002/, 01.12.14, 209
  /002/, 01.12.14, 7
  </code></pre>
  
  <p>etc etc</p>
  
  <p>In other words:</p>
  
  <pre><code>Foreach /device/
  Foreach &lt;date&gt;
  count total matching files - write number to file
  count toal matching 'not settled' files - write number to file
  </code></pre>
  
  <p>Each string to match could appear anywhere in the file.</p>
  
  <p>I tried using grep piped to a second (and third) grep commands, but I'd like to automate this and loop through the variables (6 devices, about 180 dates, 2 status strings) . I suspect Perl and Bash is the answer, but I'm out of my depth.</p>
  
  <p>Please can anyone recommend an approach to this?</p>
  
  <p>Edit: Some sample data as mentioned in the comments. The information is basically receipt data from tills - as would be sent to a printer. Here's a sample (identifying bits stripped out).</p>
  
  <pre><code>c0!                       SUBTOTAL     11.37   
  c0!   ! T O T A L      11.37! 
  c0!   19 ITEMS                                 
  c0!   C a s h             ?            11.37   
  vu p022c0!                                            
  c0!   NET TOTAL           VAT  A       10.87   
  c0!   VAT                 00.0%         0.00   
  c0!   NET TOTAL           VAT  B        0.42   
  c0!   VAT                 20.0%         0.08   
  c0!  *4300 772/080/003/132 08.01.15 11:18 A-00 
  
        Contents = Not Settled
  </code></pre>
  
  <p>In the case above, I'd be looking for /003/ , 08.01.15, and "Not Settled"</p>
  
  <p>Many thanks.</p>