title: Spark Exception: Python in worker has different version 3.4 than that in driver 3.5
body: <p>I am using Amazon EC2, and I have my master and development servers as one. And I have another instance for a single worker.</p>
  
  <p>I am new to this, but I have managed to make spark work in a standalone mode. Now I am trying cluster. the master and worker are active (I can see the webUI for them and they are functioning).</p>
  
  <p>I have Spark 2.0, and I have installed the latest Anaconda 4.1.1 which comes with Python 3.5.2. In both worker and master, if I go to pyspark and do os.version_info, I will get the 3.5.2, I also have set all the environment variables correctly (as seen in other posts on stackoverflow and google) (e.g., PYSPARK_PYTHON).</p>
  
  <p>There is no 3.4 version of python anywhere anyways. So I am wondering how I can fix this. </p>
  
  <p>I get the error by running this command:</p>
  
  <pre><code>rdd = sc.parallelize([1,2,3])
  rdd.count()    
  </code></pre>
  
  <p>error happens for the count() method:</p>
  
  <pre><code>16/08/13 18:44:31 ERROR Executor: Exception in task 1.0 in stage 2.0 (TID 17)
  org.apache.spark.api.python.PythonException: Traceback (most recent call last):
    File "/opt/spark/python/lib/pyspark.zip/pyspark/worker.py", line 123, in main
      ("%d.%d" % sys.version_info[:2], version))
  Exception: Python in worker has different version 3.4 than that in driver 3.5, PySpark cannot run with different minor versions
  
  at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRDD.scala:193)
  at org.apache.spark.api.python.PythonRunner$$anon$1.&lt;init&gt;(PythonRDD.scala:234)
  at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:152)
  at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
  at org.apache.spark.scheduler.Task.run(Task.scala:85)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)
  16/08/13 18:44:31 ERROR Executor: Exception in task 1.1 in stage 2.0 (TID 18)
  org.apache.spark.api.python.PythonException: Traceback (most recent call last):
    File "/opt/spark/python/lib/pyspark.zip/pyspark/worker.py", line 123, in main
      ("%d.%d" % sys.version_info[:2], version))
  Exception: Python in worker has different version 3.4 than that in driver 3.5, PySpark cannot run with different minor versions
  at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRDD.scala:193)
  at org.apache.spark.api.python.PythonRunner$$anon$1.&lt;init&gt;(PythonRDD.scala:234)
  at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:152)
  at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
  at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
  at org.apache.spark.scheduler.Task.run(Task.scala:85)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)
  </code></pre>