title: Spark task deserialization time
body: <p>I'm running a Spark SQL job, and when looking at the master UI, task deserialization time can take 12 seconds and the compute time 2 seconds. </p>
  
  <p>Let me give some background:<br>
  1- The task is simple: run a query in a PostgreSQL DB and count the results in Spark.<br>
  2- The deserialization problem comes when running on a cluster with 2+ workers (one of them the driver) and shipping tasks to the other worker.<br>
  3- I have to use the JDBC driver for Postgres, and I run each job with Spark submit.</p>
  
  <p>My questions: </p>
  
  <p>Am I submitting the packaged jars as part of the job every time and that is the reason for the huge task deserialization time? If so, how can I ship everything to the workers once and then in subsequent jobs already have everything needed there? </p>
  
  <p>Is there a way to keep the <code>SparkContext</code> alive between jobs (spark-submit) so that the deserialization times reduces?</p>
  
  <p>Anyways, anything that can help not paying des. time every time I run a job in a cluster.</p>
  
  <p>Thanks for your time,</p>
  
  <p>Cheers</p>