title: unix pipe is streaming?
body: <p><code>zcat big.txt.gz  |  split  -l 1000000  - prefix</code></p>
  
  <p>where <code>big.txt.gz</code> is 150 GB, say it has ~1 billion lines.</p>
  
  <p><strong>In this case, does the unix pipe "stream" the data into split, or is the  <code>zcat</code> operation completed, and then <code>split</code> is performed afterwards?</strong></p>
  
  <p>It was not clear to me from <a href="https://unix.stackexchange.com/questions/177740/what-exactly-is-the-difference-between-pipes-and-streams">other</a> <a href="https://linux.die.net/man/7/pipe" rel="nofollow noreferrer">pages</a> if the above command would crash because it couldn't hold all of the gunzipped data in the pipe buffer before executing split, or if the gunzipped data would be "streamed" into <code>split</code>.</p>
  
  <p>In general, the streaming behavior of unix <code>pipe</code> is unclear - when does pipe wait until all previous operations are finished before feeding input into the next commands stdin?</p>
  
  <p>For example, if I were to link several more commands, would it crash due to lack of memory? e.g.</p>
  
  <p><code>zcat big.txt.gz | tr 'a' 'b' | sed 's/foo/bar/g' | grep 'hello'</code></p>