title: Pass a function with any case class return type as parameter
body: <p>This might be a silly question but I've been struggling for quite some time. It is indeed similar to <a href="https://stackoverflow.com/questions/53591578/passing-case-class-into-function-arguments">this question</a> but I wasn't able to apply it in my code (duo to patterns or being a function).</p>
  
  <p>I want to pass a flatMap (or map) transform function to a function argument and then proxy it to a strategy function that actually calls the df.rdd.flatMap method. I'll try to explain!</p>
  
  <pre><code>case class Order(id: String, totalValue: Double, freight: Double) 
  case class Product(id: String, price: Double) 
  
  ... or any other case class, whatever one needs to transform a row into ...
  </code></pre>
  
  <p>The Entity class:</p>
  
  <pre><code>class Entity(path: String) = {
    ...
    def flatMap[T](mapFunction: (Row) =&gt; ArrayBuffer[T]): Entity = {
        this.getStrategy.flatMap[T](mapFunction)
        return this
    }
    def save(path: String): Unit = {
        ... write logic ...
    } 
  }
  </code></pre>
  
  <p>An Entity might have different strategies for its methods. EntityStrategy is as follows:</p>
  
  <pre><code>abstract class EntityStrategy(private val entity: Entity,
                                private val spark: SparkSession) {
    ...
    def flatMap[T](mapFunction: (Row) =&gt; ArrayBuffer[T])
    def map[T](mapFunction: (Row) =&gt; T)
  }
  </code></pre>
  
  <p>And one sample EntityStrategy implementation:</p>
  
  <pre><code>class SparkEntityStrategy(private val entity: Entity, private val spark: SparkSession)
    extends EntityStrategy(entity, spark) {
    ...
    override def map[T](mapFunction: Row =&gt; T): Unit = {
      val rdd = this.getData.rdd.map(f = mapFunction)
      this.dataFrame = this.spark.createDataFrame(rdd)
    }
  
    override def flatMap[T](mapFunction: (Row) =&gt; ArrayBuffer[T]): Unit = {
      var rdd = this.getData.rdd.flatMap(f = mapFunction)
      this.dataFrame = this.spark.createDataFrame(rdd)
    }
  }
  </code></pre>
  
  <p>Finally, I would like to create a flatMap/map function and call it like this:</p>
  
  <pre><code>def transformFlatMap(row: Row): ArrayBuffer[Order] = {
      var orders = new ArrayBuffer[Order]
      var _deliveries = row.getAs[Seq[Row]]("deliveries")
      _deliveries.foreach(_delivery =&gt; {
         var order = Order(
             id = row.getAs[String]("id"),
             totalValue = _delivery.getAs("totalAmount").asInstanceOf[Double])
        orders += order
      })
     return orders
  }
  
  val entity = new Entity("path")
  entity.flatMap[Order](transformFlatMap).save("path")
  </code></pre>
  
  <p>Of course, this doesn't work. I get an error on SparkEntityStrategy:</p>
  
  <blockquote>
    <p>Error:(95, 35) No ClassTag available for T
        val rdd = this.getData.rdd.map(f = mapFunction)</p>
  </blockquote>
  
  <p>I have tried adding an <code>(implicit encoder: Encoder: T)</code> to both entity and strategy methods but it was a no go. Probably done something wrong as I'm new to Scala.</p>
  
  <p>If I remove the "T's" and pass an actual case class everything works fine.</p>