title: Spark (scala) Unit test - Mocking an object member
body: <p>I have a spark application that involves 2 scala companion objects as follows. </p>
  
  <pre><code>object actualWorker {
    daoClient
  
    def update (data, sc) {
      groupedData = sc.getRdd(data).filter. &lt;several_operations&gt;.groupByKey
      groupedData.foreach(x =&gt; daoClient.load(x))
    }
  }
  
  
  object SparkDriver {
    getArgs
    sc = getSparkContext
    actualWorker.update(data, sc : sparkContext)
  
  }
  </code></pre>
  
  <p>The challenge I have is in writing unit-test for this spark application. I am using Mockito and ScalaTest, Junit for these tests. </p>
  
  <p>I am not able to mock the daoClient while writing the unit test. [EDIT1: Additional challenge is the fact that my daoClient is not serializable. Because I am running it on spark, I simply put it in an object (not class) and it works on spark; but it makes it non unit-testable ] </p>
  
  <p>I have tried the following:</p>
  
  <ol>
  <li>Make ActualWorker a class that can have a uploadClient passed in the
  Constructor. Create a client and instantiate it in Actual Worker
  Problem: Task not serializable exception.  </li>
  <li>Introduce a trait for upload client. But still I need to instantiate a client at some point in the SparkDriver, which I fear will cause the Task Not serializable exception.</li>
  </ol>
  
  <p>Any inputs here will be appreciated. </p>
  
  <p>PS: I am fairly new to Scala and spark</p>