title: Remove comments from beginning text file, in Ruby or otherwise
body: <p>My problem is this: I have a number of very large data files (totaling several gigabytes) which, besides the actual data, include some commented lines at the beginning of the file.</p>
  
  <p>So they look like this:</p>
  
  <pre><code># This is a file containing data.
  # Here's how to use it:
  # ...
  # Now, let's get to the actual data:
  DATA DATA DATA
  </code></pre>
  
  <p>The task I'm trying to accomplish is to remove the comments from these files as quickly as possible, because the tool I use to ingest them into my database cannot handle the comments. </p>
  
  <p>My current method is this:</p>
  
  <pre><code># For each data file...
  system "sed '/^\\#/d' #{filename} &gt; #{filename}.tmp"
  system "mv #{filename}.tmp #{filename}"
  </code></pre>
  
  <p>This works, but it takes about 1 hour in total, since the files are so large. My assumption is that there must be a more efficient way of doing this, since I know for certain that comments are <strong>only at the beginning</strong> of the files, so there's no need to search the entire file for them.</p>
  
  <p>I don't care much whether the solution uses pure Ruby, or a shell command like the one above, as long as it works much faster than the current method.</p>