title: Apache spark - Cassandra Guava incompatibility
body: <p>I am using Apache spark 2.1.0, Apache Spark connector 2.0.0-M3 and Cassandra driver core 3.0.0
  I get the following error when I try to execute the program:</p>
  
  <pre><code>17/01/19 10:38:27 WARN TaskSetManager: Lost task 1.0 in stage 1.0 (TID 5, 10.10.10.51, executor 1): java.lang.NoClassDefFoundError: Could not initialize class com.datastax.driver.core.Cluster
      at com.datastax.spark.connector.cql.DefaultConnectionFactory$.clusterBuilder(CassandraConnectionFactory.scala:35)
      at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:92)
      at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:154)
      at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
      at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149)
      at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31)
      at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56)
      at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:82)
      at com.datastax.spark.connector.rdd.CassandraTableScanRDD.compute(CassandraTableScanRDD.scala:326)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
      at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
      at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
      at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
      at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
      at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
      at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
      at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
      at org.apache.spark.scheduler.Task.run(Task.scala:99)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745) 
  
  Caused by: java.lang.IllegalStateException: Detected Guava issue #1635 which indicates that a version of Guava less than 16.01 is in use.  This introduces codec resolution issues and potentially other incompatibility issues in the driver.  Please upgrade to Guava 16.01 or later.
      at com.datastax.driver.core.SanityChecks.checkGuava(SanityChecks.java:62)
      at com.datastax.driver.core.SanityChecks.check(SanityChecks.java:36)
      at com.datastax.driver.core.Cluster.&lt;clinit&gt;(Cluster.java:67)
  </code></pre>
  
  <p>I have read that spark-cassandra-connector solves the guava compatibility with spark problem:</p>
  
  <p><a href="https://groups.google.com/a/lists.datastax.com/forum/#!topic/spark-connector-user/aIpytzBAen4" rel="nofollow noreferrer">https://groups.google.com/a/lists.datastax.com/forum/#!topic/spark-connector-user/aIpytzBAen4</a></p>
  
  <p>I am executing with SparkMaster api 7077 with JettyRun and ClusterMode. I pass the jars of the cassandra driver and spark-cassandra connector to the spark conf (setJars)</p>
  
  <p>Some times if I restart it works, but several times I have to try and try and never works.</p>
  
  <p>I tried some answers like rename the Spark guava jar to 19 version but always got the same problem. </p>
  
  <p>What is happening?</p>