title: Jupyter + EMR + Spark - Connect to EMR cluster from Jupyter notebook on local machine
body: <p>I am new to PySpark and EMR.<br />
  I am trying to access Spark running on EMR cluster through Jupyter notebook, but running into errors.</p>
  <p>I am generating SparkSession using following code:</p>
  <pre><code>spark = SparkSession.builder \
      .master(&quot;local[*]&quot;)\
      .appName(&quot;parallelization on Spark&quot;)\
      .getOrCreate()
  </code></pre>
  <p>Tried following to access Remote cluster, but it errored out:</p>
  <pre><code>spark = SparkSession.builder \
      .master(&quot;spark://&lt;remote-emr-ec2-hostname&gt;:7077&quot;)\
      .appName(&quot;parallelization on Spark&quot;)\
      .getOrCreate()
  </code></pre>
  <p>Error:</p>
  <pre><code>Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
  : java.lang.NullPointerException
      at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:567)
      at org.apache.spark.api.java.JavaSparkContext.&lt;init&gt;(JavaSparkContext.scala:58)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  </code></pre>
  <p>Any help resolving this would be much appreciated.</p>