title: pyspark cannot generate job when reading big table by sqlquery
body: <p>I use pyspark to read a big table, but I cannot find running job in webui's Jobs tab.
  In webui's Executors tab, only 2 executors in active. Others are dead.
  In webui's SQL tab, have a running Query.The query always continuing almost one hour then the job will running.
  <a href="https://i.stack.imgur.com/Y8B1t.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Y8B1t.png" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/ixapL.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ixapL.png" alt="enter image description here" /></a>
  code as below:</p>
  <pre><code>from pyspark import SparkConf, SparkContext
  from pyspark.sql import SparkSession
  import pyspark.sql.functions as F
  from pyspark.sql.window import Window as W
  from pyspark.sql.types import *
  from pyspark.ml.feature import StringIndexer, VectorAssembler
  from pyspark.ml import Pipeline
  from pyspark.sql.functions import col
  
  spark = SparkSession.builder.\
          config('spark.executor.memory', '12g').\
          config('spark.executor.cores', '3').\
          config('spark.driver.memory','16g').\
          config('spark.executor.instances', '60').\
          config('spark.driver.maxResultSize', '50g').\
          config(&quot;spark.driver.cores&quot;, '4').\
          config('spark.default.parallelism', '360').\
          config('spark.task.cpus', '1').\
          config('hive.exec.orc.split.strategy','ETL').\
          config(&quot;spark.yarn.queue&quot;,&quot;DataBusinessDept_Risk_Query&quot;).\
          config(&quot;spark.hadoop.hive.exec.dynamic.partition&quot;, &quot;true&quot;).\
          config(&quot;spark.hadoop.hive.exec.dynamic.partition.mode&quot;, &quot;nonstrict&quot;).\
          config('spark.rpc.message.maxSize', '512').\
          config('spark.sql.sources.partitionOverwriteMode', 'dynamic').\
          config('spark.sql.getSplit.parallelism.num', '180').\
          appName('test').\
          enableHiveSupport().getOrCreate()
  
  a = spark.sql(&quot;&quot;&quot;
          select t2.*
            from table_name t2
           where pt between '20210401' and '20210416'
  &quot;&quot;&quot;).cache()
  a.count()
  </code></pre>