title: How to get full worker output in Apache Spark
body: <p>How do I view / download the complete stderr output from a worker in Apache Spark, deployed in cluster mode?</p>
  <p>I've deployed a program with <code>spark-submit --deploy-mode cluter foo.jar</code>, and a worker crashed.  To investigate, I go to <code>localhost:8081</code> and access the worker's log (stderr in particular), but then it shows me only the bottom of the file, and I have to click the &quot;Load More&quot; button a hundred times to scroll up to the first error -- clearly, I shouldn't have to do that.  Is there a way to download the whole stderr output, or to redirect it to a known location?  Which part of Spark's documentation gives me this kind of information?</p>