 <p>I totally agree with @Holden on that! </p> <p>Mocking RDDS is difficult; executing your unit tests in a local Spark context is preferred, as recommended in the <a href="http://spark.apache.org/docs/latest/programming-guide.html#unit-testing">programming guide</a>.</p> <p>I know this may not technically be a unit test, but it is hopefully close enough. </p> <blockquote> <p><strong>Unit Testing</strong></p> <p>Spark is friendly to unit testing with any popular unit test framework. Simply create a SparkContext in your test with the master URL set to local, run your operations, and then call SparkContext.stop() to tear it down. Make sure you stop the context within a finally block or the test framework’s tearDown method, as Spark does not support two contexts running concurrently in the same program.</p> </blockquote> <p>But if you are really interested and you still want to try mocking RDDs, I'll suggest that you read the <a href="https://github.com/apache/spark/blob/3c0156899dc1ec1f7dfe6d7c8af47fa6dc7d00bf/core/src/test/scala/org/apache/sparktest/ImplicitSuite.scala">ImplicitSuite</a> test code.</p> <p>The only reason they are pseudo-mocking the RDD is to test if <code>implict</code> works well with the compiler, but they don't actually need a real RDD.</p> <pre><code>def mockRDD[T]: org.apache.spark.rdd.RDD[T] = null </code></pre> <p>And it's not even a real mock. It just creates a null object of type RDD[T]</p>