title: pyspark: Could not find valid SPARK_HOME
body: <p>After a fresh installation of <code>pyspark</code> via  <code>pip install pyspark</code> I get the following errors:</p>
  
  <pre><code>&gt; pyspark
  Could not find valid SPARK_HOME while searching ['/Users', '/usr/local/bin']
  /usr/local/bin/pyspark: line 24: /bin/load-spark-env.sh: No such file or directory
  /usr/local/bin/pyspark: line 77: /bin/spark-submit: No such file or directory
  /usr/local/bin/pyspark: line 77: exec: /bin/spark-submit: cannot execute: No such file or directory
  &gt; spark-shell
  Could not find valid SPARK_HOME while searching ['/Users', '/usr/local/bin']
  /usr/local/bin/spark-shell: line 57: /bin/spark-submit: No such file or directory
  </code></pre>
  
  <p>What is a valid <code>SPARK_HOME</code>, how do I set it, and why is there no default that works? </p>
  
  <p>I've seen instructions on how to manually set the environment variable after a manual installation of spark, but I am wondering how to set it in this case after using <code>pip</code> to install <code>pyspark</code>.  </p>
  
  <p>I've installed only spark via <code>brew install apache-spark</code>, and the <code>spark-shell</code> that comes out of that installation works just fine out of the box. After installing pyspark after that, I get the messages above. Confusing.</p>