title: Binary grep on Linux?
body: <p>Say I have generated the following binary file:</p>
  
  <pre><code># generate file:
  python -c 'import sys;[sys.stdout.write(chr(i)) for i in (0,0,0,0,2,4,6,8,0,1,3,0,5,20)]' &gt; mydata.bin
  
  # get file size in bytes
  stat -c '%s' mydata.bin
  
  # 14
  </code></pre>
  
  <p>And say, I want to find the locations of all zeroes (<code>0x00</code>), using a grep-like syntax.</p>
  
  <p>&nbsp;</p>
  
  <p>The best I can do so far is:</p>
  
  <pre><code>$ hexdump -v -e "1/1 \" %02x\n\"" mydata.bin | grep -n '00'
  
  1: 00
  2: 00
  3: 00
  4: 00
  9: 00
  12: 00
  </code></pre>
  
  <p>However, this implicitly converts each byte in the original binary file into a multi-byte ASCII representation, on which <code>grep</code> operates; not exactly the prime example of optimization :)</p>
  
  <p>Is there something like a binary <code>grep</code> for Linux? Possibly, also, something that would support a regular expression-like syntax, but also for byte "characters" - that is, I could write something like '<code>a(\x00*)b</code>' and match 'zero or more' occurrences of byte 0 between bytes 'a' (97) and 'b' (98)?</p>
  
  <p>EDIT: The context is that I'm working on a driver, where I capture 8-bit data; something goes wrong in the data, which can be kilobytes up to megabytes, and I'd like to check for particular signatures and where they occur. (<em>so far, I'm working with kilobyte snippets, so optimization is not that important - but if I start getting some errors in megabyte long captures, and I need to analyze those, my guess is I would like something more optimized :) . And especially, I'd like something where I can "grep" for a byte as a character - <code>hexdump</code> forces me to search strings per byte</em>)</p>
  
  <p>EDIT2: same question, different forum :) <a href="http://www.linuxquestions.org/questions/linux-software-2/grepping-through-a-binary-file-for-a-sequence-of-bytes-717503/" rel="noreferrer">grepping through a binary file for a sequence of bytes</a></p>
  
  <p>EDIT3: Thanks to the answer by @tchrist, here is also an example with 'grepping' and matching, and displaying results (<em>although not quite the same question as OP</em>):</p>
  
  <pre><code>$ perl -ln0777e 'print unpack("H*",$1), "\n", pos() while /(.....\0\0\0\xCC\0\0\0.....)/g' /path/to/myfile.bin
  
  ca000000cb000000cc000000cd000000ce     # Matched data (hex)
  66357                                  # Offset (dec)
  </code></pre>
  
  <p>To have the matched data be grouped as one byte (two hex characters) each, then "H2 H2 H2 ..." needs to be specified for as many bytes are there in the matched string; as my match '<code>.....\0\0\0\xCC\0\0\0.....</code>' covers 17 bytes, I can write '<code>"H2"x17</code>' in Perl. Each of these "H2" will return a separate variable (as in a list), so <code>join</code> also needs to be used to add spaces between them - eventually:</p>
  
  <pre><code>$ perl -ln0777e 'print join(" ", unpack("H2 "x17,$1)), "\n", pos() while /(.....\0\0\0\xCC\0\0\0.....)/g' /path/to/myfile.bin
  
  ca 00 00 00 cb 00 00 00 cc 00 00 00 cd 00 00 00 ce
  66357
  </code></pre>
  
  <p>Well.. indeed Perl is very nice 'binary grepping' facility, I must admit :) As long as one learns the syntax properly :)</p>