title: Proper way to make a Spark Fat Jar using SBT
body: <p>I need a Fat Jar with Spark because I'm creating a custom node for Knime. Basically it's a self-contained jar executed inside Knime and I assume a Fat Jar is the only way to spawn a local Spark Job. Eventually we will go on submitting a job to a remote cluster but for now I need it to spawn this way.</p>
  
  <p>That said, I made a Fat Jar using this: <a href="https://github.com/sbt/sbt-assembly" rel="noreferrer">https://github.com/sbt/sbt-assembly</a></p>
  
  <p>I made an empty sbt project, included Spark-core in the dependencies and assembled the Jar. I added it to the manifest of my custom Knime node and tried to spawn a simple job (pararellize a collection, collect it and print it). It starts but I get this error:</p>
  
  <blockquote>
    <p>No configuration setting found for key 'akka.version'</p>
  </blockquote>
  
  <p>I have no idea how to solve it. </p>
  
  <p>Edit: this is my build.sbt</p>
  
  <pre><code>name := "SparkFatJar"
  
  version := "1.0"
  
  scalaVersion := "2.11.6"
  
  libraryDependencies ++= Seq(
    "org.apache.spark" %% "spark-core" % "1.3.0"
  )
  
  
  libraryDependencies +=  "com.typesafe.akka" %% "akka-actor" % "2.3.8"
  
  assemblyJarName in assembly := "SparkFatJar.jar"
  
  assemblyMergeStrategy in assembly := {
    case PathList("META-INF", xs @ _*) =&gt; MergeStrategy.discard
    case x =&gt; MergeStrategy.first
  }
  </code></pre>
  
  <p>I've found this mergestrategy for Spark somewhere on the internet but I can't find the source right now.</p>