title: How to elegantly implement the pipeline pattern using Scala
body: <p>I'm looking to build a pipeline pattern with Scala. I wish after I write the pipeline objects, they could be connected together like this:</p>
  
  <pre><code>Pipeline1 :: Pipeline2 :: Pipeline3 ...
  </code></pre>
  
  <p>I have experimented with a few ideas so far. Some work and some don't. But none of them seems to completely get rid of boilerplate code. The following is the closest I've got.</p>
  
  <p>First define the Pipeline and Source abstract class:</p>
  
  <pre><code>// I is the input type and O is the output type of the pipeline
  abstract class Pipeline[I, +O](p: Pipeline[_, _ &lt;: I]) {
  
    val source = p
    val name: String
    def produce(): O
    def stats():String
  }
  abstract class Source[+T] extends Pipeline[AnyRef, T](null)
  </code></pre>
  
  <p>Next, I created two pipelines and try to link them together</p>
  
  <pre><code>// this creates a random integer
  class RandomInteger extends Source[Int] {
    override val name = "randInt"
  
    def produce() = {
      scala.Math.round(scala.Math.random.asInstanceOf[Float] * 10)
    }
  
    def stats()="this pipeline is stateless"
  }
  
  // multiply it by ten
  class TimesTen(p: Pipeline[_, Int]) extends Pipeline[Int, Int](p) {
    private var count = 0 // this is a simple state of the pipeline
    override val name = "Times"
    def produce = {
      val i = source.produce()
      count += 1 // updating the state
      i * 10
    }
    def stats() = "this pipeline has been called for " + count + " times"
  }
  
  object TimesTen {
    // this code achieves the desired connection using ::
    // but this has to be repeated in each pipeline subclass. 
    // how to remove or abstract away this boilerplate code? 
    def ::(that: Pipeline[_, Int]) = new TimesTen(that)
  }
  </code></pre>
  
  <p>This is the main class where two pipelines are linked.</p>
  
  <pre><code>object Pipeline {
    def main(args: Array[String]) {
      val p = new RandomInteger() :: TimesTen
      println(p.source)
      for (i &lt;- 0 to 10)
        println(p.produce())
      println(p.stats())
    }
  }
  </code></pre>
  
  <p>So this code works. But I would have to repeat the code in the TimesTen companion object in every pipeline class I write. This is certainly not desirable. Is there any better way to do this? Reflection might work, but I heard bad things about it, such as anything involving reflection is bad design. I'm also unsure about Scala's support for reflection.</p>
  
  <p>Thank you for your time.</p>
  
  <p><strong>Update</strong>: I designed this toy problem to make it easy to understand. As a general solution, and as my application requires, each pipeline object has a state, which is ideally encapsulated within the object itself rather than exposed to every other pipeline. I have modified the code above to reflect this. I wish there could be an object-based solution. I'm still experimenting and will let you know if I find one.</p>
  
  <p><strong>Update 2</strong>: After some thoughts, I think the idea of the pipeline is really just a generalized function that contains some internal states as well as the ability to compose a <code>Function0</code> function with a <code>Function1</code> function. In Scala, the <code>Function0</code> class does not have the <code>compose()</code> or <code>andThen()</code> method. </p>