title: Is the stream splitting from bash transposable to other languages?
body: <p>By <em>stream splitting</em> I mean the ability to:</p>
  
  <ol>
  <li>filter on the fly the stream content by a first function</li>
  <li>one part of the stream is processed by a second function</li>
  <li>the rest of the stream is processed by a third function</li>
  <li>the stream is never stored (on the fly)</li>
  </ol>
  
  <p>An example is sometime better than a long explanation. This command line use <a href="http://en.wikipedia.org/wiki/Tee_%28command%29" rel="nofollow noreferrer"><code>tee</code></a> and <a href="http://en.wikipedia.org/wiki/Process_substitution" rel="nofollow noreferrer">process substitution</a> to split the stream:</p>
  
  <pre><code>$&gt; cut -f2 file | tee &gt;( grep "AB" | sort | ... ) | grep -v "AB" | tr A B | ...
  </code></pre>
  
  <p>In this example, the stream is split in two: the lines containing <code>"AB"</code> and the rest:</p>
  
  <pre><code>cut -f2 file ----&gt;- line contains "AB" -&gt;- sort -&gt;- ...
               \---&gt;- does not contain "AB" -&gt;- tr A B -&gt;- ...
  </code></pre>
  
  <p>But I do not like this stream splitting technique because the stream is first duplicated (by <code>tee</code>) to be then filtered twice (by <code>grep</code> and <code>grep -v</code>). </p>
  
  <p>Therefore I wonder if something like <em>stream splitting</em> is available in other languages as <a href="/questions/tagged/perl" class="post-tag" title="show questions tagged &#39;perl&#39;" rel="tag">perl</a>, <a href="/questions/tagged/python" class="post-tag" title="show questions tagged &#39;python&#39;" rel="tag">python</a>, <a href="/questions/tagged/ruby" class="post-tag" title="show questions tagged &#39;ruby&#39;" rel="tag">ruby</a>, <a href="/questions/tagged/c%2b%2b" class="post-tag" title="show questions tagged &#39;c++&#39;" rel="tag">c++</a>... </p>
  
  <p>I provide a more complex example below.</p>
  
  <hr>
  
  <h2>Complex <code>bash</code> stream splitting</h2>
  
  <p><code>counter.sh</code> splits a stream in three sections (begin, middle and end). And for each section, the stream is again split to count the occurrences of symbols <code>&lt;</code>, <code>|</code> and <code>&gt;</code>:</p>
  
  <pre><code>#!/bin/bash    
  {
    {  tee &gt;( sed -n '1,/^--$/!p' &gt;&amp;3 ) |
              sed -n '1,/^--$/p'        |
       tee &gt;( echo "del at begin:  $(grep -c '&lt;')"    &gt;&amp;4 ) |
       tee &gt;( echo "add at begin:  $(grep -c '&gt;')"    &gt;&amp;4 ) |
            { echo "chg at begin:  $(grep -c '|')"; } &gt;&amp;4
    }  3&gt;&amp;1 1&gt;&amp;2  |
    {  tee &gt;( sed -n '/^--$/,/^--$/!p' &gt;&amp;3 ) |
              sed -n '/^--$/,/^--$/p'        |
       tee &gt;( echo "del at end:    $(grep -c '&lt;')"    &gt;&amp;4 ) |
       tee &gt;( echo "add at end:    $(grep -c '&gt;')"    &gt;&amp;4 ) |
            { echo "chg at end:    $(grep -c '|')"; } &gt;&amp;4
    }  3&gt;&amp;1 1&gt;&amp;2 |
       tee &gt;( echo "del in middle: $(grep -c '&lt;')"    &gt;&amp;4 ) |
       tee &gt;( echo "add in middle: $(grep -c '&gt;')"    &gt;&amp;4 ) |
              echo "chg in middle: $(grep -c '|')"; 
  } 4&gt;&amp;1
  </code></pre>
  
  <p>This script is used to count the number of added/changed/deleted lines in sections begin/middle/end. The input of this script is a stream:</p>
  
  <pre><code>$&gt; cat file-A
  1
  22
  3
  4
  5
  6
  77
  8
  
  $&gt; cat file-B
  22
  3
  4
  42
  6
  77
  8
  99
  
  $&gt; diff --side-by-side file-A file-B | egrep -1 '&lt;|\||&gt;' | ./counter.sh
  del at begin:  1
  add at begin:  0
  chg at begin:  0
  del at end:    0
  add at end:    1
  chg at end:    0
  del in middle: 0
  add in middle: 0
  chg in middle: 1
  </code></pre>
  
  <p>How to implement efficiently a such <code>counter.sh</code> in other programming languages without storing the data in a temporary buffer?</p>
  
  <hr>
  
  <h2>Answer</h2>
  
  <p>As noted by <a href="https://stackoverflow.com/users/126214">Lennart Regebro</a>, I am <em>over-thinking</em> this question. Of course, all these languages are able to split input streams as answered by <a href="https://stackoverflow.com/a/19638354/938111">ysth</a>. In pseudo code:</p>
  
  <pre><code>while input-stream
  {
      case (begin section)
      {
          case (symbol &lt;) aB++ 
          case (symbol |) cB++ 
          case (symbol &gt;) dB++
      }
      case (middle section)
      {
          case (symbol &lt;) aM++ 
          case (symbol |) cM++ 
          case (symbol &gt;) dM++
      } 
      case (ending section)
      {
          case (symbol &lt;) aE++ 
          case (symbol |) cE++ 
          case (symbol &gt;) dE++
      }
  }
  
  PrintResult (aB, cB, dB, aM, cM, dM, aE, cE, dE)
  </code></pre>
  
  <p><strong>Conclusion:</strong> Stream splitting is better done in <code>python</code>/<code>perl</code>/<code>awk</code>/<code>C++</code> than using <a href="http://en.wikipedia.org/wiki/Tee_%28command%29" rel="nofollow noreferrer"><code>tee</code></a> + <a href="http://en.wikipedia.org/wiki/Process_substitution" rel="nofollow noreferrer">process substitution</a>.</p>