title: Spark-submit executor memory issue
body: <p>I have a 10 node cluster, 8 DNs(256 GB, 48 cores) and 2 NNs. I have a spark sql job being submitted to the yarn cluster. Below are the parameters which I have used for spark-submit.
  --num-executors 8 \
  --executor-cores 50 \
  --driver-memory 20G \
  --executor-memory 60G \
  As can be seen above executor-memory is 60GB, but when I check Spark UI is shows 31GB.
  <a href="https://i.stack.imgur.com/ATZc9.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ATZc9.jpg" alt="enter image description here"></a></p>
  
  <p>1) Can anyone explain me why it is showing 31GB instead of 60GB.
  2) Also help in setting optimal values for parameters mentioned above.</p>