title: Apache Spark: NoSuchMethodError
body: <p>i'm trying to do some tests on Apache Spark (v1.3.0), i have a simple Java 8 class:</p>
  
  <pre><code>public class WordCount {
      private JavaSparkContext ctx;
      private String inputFile, outputFile;
      public WordCount(String inputFile, String outputFile) {
          this.inputFile = inputFile;
          this.outputFile = outputFile;
          // Initialize Spark Conf
          ctx = new JavaSparkContext("local", "WordCount",
                  System.getenv("SPARK_HOME"), System.getenv("JARS"));
  
      }
  
      public static void main(String... args) {
          String inputFile = "/home/workspace/spark/src/main/resources/inferno.txt";//args[0];
          String outputFile = "/home/workspace/spark/src/main/resources/dv";//args[1];
          WordCount wc = new WordCount(inputFile, outputFile);
          wc.doWordCount();
          wc.close();
      }
  
      public void doWordCount() {
          long start = System.currentTimeMillis();
          JavaRDD&lt;String&gt; inputRdd = ctx.textFile(inputFile);
          JavaPairRDD&lt;String, Integer&gt; count = inputRdd.flatMapToPair((String s) -&gt; {
              List&lt;Tuple2&lt;String, Integer&gt;&gt; list = new ArrayList&lt;&gt;();
              Arrays.asList(s.split(" ")).forEach(s1 -&gt; list.add(new Tuple2&lt;String, Integer&gt;(s1, 1)));
              return list;
          }).reduceByKey((x, y) -&gt; x + y);
          Tuple2&lt;String, Integer&gt; max = count.max(new Tuple2Comparator());
          System.out.println(max);
  //      count.saveAsTextFile(outputFile);
          long end = System.currentTimeMillis();
          System.out.println(String.format("Time in ms is: %d", end - start));
      }
  
      public void close() {
          ctx.stop();
      }
  
  }
  </code></pre>
  
  <p>The Tuple2Comparator comparator class is:</p>
  
  <pre><code>public class Tuple2Comparator implements Comparator&lt;Tuple2&lt;String, Integer&gt;&gt;, Serializable {
      private static final long serialVersionUID = 103955884403243585L;
  
      @Override
      public int compare(Tuple2&lt;String, Integer&gt; o1, Tuple2&lt;String, Integer&gt; o2) {
          return o2._2() - o1._2();
      }
  
  }
  </code></pre>
  
  <p>When i run it get the following exception:</p>
  
  <pre><code>Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.api.java.JavaPairRDD.max(Ljava/util/Comparator;)Lscala/Tuple2;
          at it.conker.spark.base.WordCount.doWordCount2(WordCount.java:69)
          at it.conker.spark.base.WordCount.main(WordCount.java:41)
  </code></pre>
  
  <p>This is mi pom file:</p>
  
  <pre><code>&lt;project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
      &lt;groupId&gt;it.conker.spark&lt;/groupId&gt;
      &lt;artifactId&gt;learning-spark-by-example&lt;/artifactId&gt;
      &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
      &lt;name&gt;Learning Spark by example&lt;/name&gt;
      &lt;packaging&gt;jar&lt;/packaging&gt;
      &lt;version&gt;0.0.1&lt;/version&gt;
      &lt;dependencies&gt;
          &lt;dependency&gt; &lt;!-- Spark dependency --&gt;
              &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
              &lt;artifactId&gt;spark-core_2.10&lt;/artifactId&gt;
              &lt;version&gt;1.3.0&lt;/version&gt;
              &lt;scope&gt;provided&lt;/scope&gt;
          &lt;/dependency&gt;
  
          &lt;dependency&gt;
              &lt;groupId&gt;junit&lt;/groupId&gt;
              &lt;artifactId&gt;junit&lt;/artifactId&gt;
              &lt;version&gt;4.11&lt;/version&gt;
              &lt;scope&gt;test&lt;/scope&gt;
          &lt;/dependency&gt;
      &lt;/dependencies&gt;
      &lt;properties&gt;
          &lt;java.version&gt;1.8&lt;/java.version&gt;
      &lt;/properties&gt;
      &lt;build&gt;
          &lt;pluginManagement&gt;
              &lt;plugins&gt;
                  &lt;plugin&gt;
                      &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
                      &lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
                      &lt;version&gt;3.1&lt;/version&gt;
                      &lt;configuration&gt;
                          &lt;source&gt;${java.version}&lt;/source&gt;
                          &lt;target&gt;${java.version}&lt;/target&gt;
                      &lt;/configuration&gt;
                  &lt;/plugin&gt;
              &lt;/plugins&gt;
          &lt;/pluginManagement&gt;
      &lt;/build&gt;
  &lt;/project&gt;
  </code></pre>
  
  <p>I run the class within eclipse.
  Can someone tell me where i was wrong?</p>