title: Presto on (Py)Spark - setup
body: <p>Through my job as a data engineer most of my data logic is through:</p>
  <ol>
  <li><p>presto queries (small - medium data sets and calculation, easiest for working with analysts)</p>
  </li>
  <li><p>Spark (the big guns, when the calculations are pretty heavy), mostly in python environment.</p>
  </li>
  </ol>
  <p>Although there are several scenarios I prefer to work smarter with presto (data sketch functions) then utilize spark.</p>
  <p>The scenario I'm looking for is to integrate presto framework with pyspark. and Im failing to setup it correctly in the session level.
  tried to convert the spark-submit example in presto docs to a pyspark session builder without any luck.</p>
  <p>/spark/bin/spark-submit <br />
  --master spark://spark-master:7077 <br />
  --executor-cores 4 <br />
  --conf spark.task.cpus=4 <br />
  --class com.facebook.presto.spark.launcher.PrestoSparkLauncher <br />
  presto-spark-launcher-0.271.jar <br />
  --package presto-spark-package-0.271.tar.gz <br />
  --config /presto/etc/config.properties <br />
  --catalogs /presto/etc/catalogs <br />
  --catalog hive <br />
  --schema default <br />
  --file query.sql</p>
  <p>loaded the jsons successfully:
  <a href="https://i.stack.imgur.com/8098P.png" rel="nofollow noreferrer">config snap</a></p>
  <p>but when trying to run spark.sql with presto syntax I still fail.
  What am I missing?</p>
  <p>I was following this documentation:
  <a href="https://prestodb.io/docs/current/installation/spark.html" rel="nofollow noreferrer">https://prestodb.io/docs/current/installation/spark.html</a></p>
  <p>This official GIT:
  <a href="https://github.com/prestodb/presto/issues/13856" rel="nofollow noreferrer">https://github.com/prestodb/presto/issues/13856</a></p>
  <p>And tried to get more data across the web (there isn't much)
  <a href="https://medium.com/@ravishankar.nair/the-ultimate-duo-in-distributed-computing-prestodb-running-on-spark-b63d0e567eeb" rel="nofollow noreferrer">https://medium.com/@ravishankar.nair/the-ultimate-duo-in-distributed-computing-prestodb-running-on-spark-b63d0e567eeb</a></p>