title: oozie shell action with spark-submit
body: <p>i am trying to run spark-submit from a shell wrapper. while the job runs fine from command line but failed when scheduling through oozie. </p>
  
  <pre><code>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
  at org.apache.spark.deploy.SparkSubmitArguments.handle(SparkSubmitArguments.scala:394)
  at org.apache.spark.launcher.SparkSubmitOptionParser.parse(SparkSubmitOptionParser.java:163)
  at org.apache.spark.deploy.SparkSubmitArguments.(SparkSubmitArguments.scala:97)
  at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:114)
  at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  </code></pre>
  
  <p>here is my workflow:</p>
  
  <pre><code>    &lt;workflow-app name="OozieTest1" xmlns="uri:oozie:workflow:0.5"&gt;
      &lt;start to="CopyTest"/&gt;
     &lt;kill name="Kill"&gt;
          &lt;message&gt;Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]&lt;/message&gt;
      &lt;/kill&gt;
  &lt;action name="CopyTest"&gt;
          &lt;shell xmlns="uri:oozie:shell-action:0.1"&gt;
              &lt;job-tracker&gt;${jobTracker}&lt;/job-tracker&gt;
              &lt;name-node&gt;${nameNode}&lt;/name-node&gt;
              &lt;exec&gt;lib/copy.sh&lt;/exec&gt;
                &lt;argument&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/file-list/xxx_xxx_201610.lst&lt;/argument&gt;
                &lt;argument&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/sample&lt;/argument&gt;
                &lt;argument&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/output&lt;/argument&gt;
                &lt;argument&gt;IMMUN&lt;/argument&gt;
                &lt;argument&gt;N&lt;/argument&gt;
                &lt;argument&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/resources/script-constants.properties&lt;/argument&gt;
               &lt;file&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/lib/copy.sh#copy.sh&lt;/file&gt; 
               &lt;file&gt;hdfs://xxxxxx/user/xxxxxx/oozie-test/lib/xxxx_Integration.jar#xxxx_Integration.jar&lt;/file&gt;
          &lt;capture-output/&gt;
          &lt;/shell&gt;
          &lt;ok to="End"/&gt;
          &lt;error to="Kill"/&gt;
      &lt;/action&gt;
      &lt;end name="End"/&gt;
  &lt;/workflow-app&gt;
  </code></pre>