title: Can't make action calls through anaconda py35 env in spark HdInsight
body: <p>As per the documentation - <a href="https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-python-package-installation" rel="nofollow noreferrer">https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-python-package-installation</a></p>
  
  <p>we had installed several external python modules through new anaconda env 'py35_data_prof'. However as soon as we invoke any rdd action calls like rdd.count() or rdd.avg() in our python code, spark2 throws -</p>
  
  <pre><code>Cannot run program "/usr/bin/anaconda/envs/py35_data_prof/bin/python": error=2, No such file or directory
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/8Y2HH.png" rel="nofollow noreferrer">enter image description here</a></p>
  
  <p>FYI, The python indicated in error path - '/usr/bin/anaconda/envs/py35_data_prof/bin/python' is actually a symlink rather than python dir.
  I have been looking up the HDInsight docs but can't seem to find the fix. Please let us know if there is a way around it.</p>