title: processing a headered CSV file with gnu parallel
body: <p>Is it possible to invoke gnu parallel in a way that it would repeat the first line of original input to the STDIN of each child job?</p>
  
  <p>I have a CSV file that contains a header line at the top. For example:</p>
  
  <pre><code>&gt; cat large.csv
  id,count
  abc,123
  def,456
  </code></pre>
  
  <p>I have a tool that can extract columns by name rather than position:</p>
  
  <pre><code>&gt; csv_extract large.csv count
  123
  456
  </code></pre>
  
  <p>I can sum the values serially as:</p>
  
  <pre><code>&gt; csv_extract large.csv count | awk '{ SUM += $1 } END { print SUM }'
  579
  </code></pre>
  
  <p>The actual file I have is much larger, and the operation more complex than summing, but the same principles would apply. I'd like to use gnu parallel to process the file, but I don't know if it is possible to tell gnu parallel to repeat the CSV header for each job.</p>
  
  <p>Ideally I could run the operation with something like:</p>
  
  <pre><code>&gt; cat large.csv | parallel --pipe --repeat-first-line "csv_extract /dev/stdin count | awk '{ SUM += $1 } END { print SUM }'"
  579
  </code></pre>
  
  <p>I've made up the <b>--repeat-first-line</b> option above to represent the functionality I cannot figure out. I've watched the YouTube videos, and read the man page, but I'm just not able to see how it can be done, if at all possible.</p>
  
  <p>Thanks!</p>
  
  <ul>
  <li>danboo</li>
  </ul>