title: Sqoop job running forever using Oozie
body: <p>We have two node hadoop yarn cluster, it is hadoop 2.2, on which We have scheduled two actions in a single workflow using oozie, the first action is a <strong><em>python map-reduce streaming action</em></strong>, and the second one is <strong><em>sqoop export</em></strong> job, which actually transfers the output of map-reduce streaming action to mysql database.</p>
  
  <p>The streaming action successfully gets executed,which results in the starting of sqoop job, which remains running for ever.</p>
  
  <p><strong>stdout</strong> results in the following.</p>
  
  <pre><code>Sqoop command arguments :
           export
           --connect
           jdbc:mysql://localhost/database
           --username
           root
           --password
           root
           --table
           tableName
           --direct
           --export-dir
          /user/hduser/oozieProject/workflow/output
  
   =================================================================
  Invoking Sqoop command line now &gt;&gt;&gt;
  
  2137 [main] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
  2158 [main] INFO  org.apache.sqoop.Sqoop  - Running Sqoop version: 1.4.4.2.0.6.1-102
  2170 [main] WARN  org.apache.sqoop.tool.BaseSqoopTool  - Setting your password on the command-line is insecure. Consider using -P instead.
  2178 [main] WARN  org.apache.sqoop.ConnFactory  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
  2197 [main] INFO  org.apache.sqoop.manager.MySQLManager  - Preparing to use a MySQL streaming resultset.
  2197 [main] INFO  org.apache.sqoop.tool.CodeGenTool  - Beginning code generation
  2464 [main] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement:     SELECT t.* FROM `missedCalls` AS t LIMIT 1
  2483 [main] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: SELECT t.* FROM `missedCalls` AS t LIMIT 1
  2485 [main] INFO  org.apache.sqoop.orm.CompilationManager  - HADOOP_MAPRED_HOME is /usr/local/hadoop
  3838 [main] INFO  org.apache.sqoop.orm.CompilationManager  - Writing jar file: /tmp/sqoop-hduser/compile/21bd1d5fe13adeed4f46a09f8b3d38fe/missedCalls.jar
  3847 [main] INFO  org.apache.sqoop.mapreduce.ExportJobBase  - Beginning export of missedCalls
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  Heart beat
  </code></pre>
  
  <p>the job properties is as follows</p>
  
  <pre><code>nameNode=hdfs://master:54310
  jobTracker=master:8035
  queueName=default
  
  oozie.libpath=${nameNode}/user/hduser/share/lib
  oozie.use.system.libpath=true
  oozie.wf.rerun.failnodes=true
  
  oozieProjectRoot=${nameNode}/user/hduser/oozieProject
  appPath=${oozieProjectRoot}/workflow
  oozie.wf.application.path=${appPath}
  oozieLibPath=${oozie.libpath}
  
  mapred.tasktracker.map.tasks.maximum=4
  mapred.tasktracker.reduce.tasks.maximum=4
  
  inputDir=${oozieProjectRoot}/data/*
  outputDir=${appPath}/output
  </code></pre>
  
  <p>the workflow xml is as follows</p>
  
  <pre><code>&lt;!--Oozie workflow file: workflow.xml --&gt; 
  &lt;workflow-app name="WorkflowStreamingMRAction-Python" xmlns="uri:oozie:workflow:0.1"&gt;
  &lt;start to="streamingaAction"/&gt;
  &lt;action name="streamingaAction"&gt;
      &lt;map-reduce&gt;
          &lt;job-tracker&gt;${jobTracker}&lt;/job-tracker&gt;
          &lt;name-node&gt;${nameNode}&lt;/name-node&gt;
          &lt;prepare&gt;
              &lt;delete path="${outputDir}"/&gt;
          &lt;/prepare&gt;
          &lt;streaming&gt;
              &lt;mapper&gt;python mapper.py&lt;/mapper&gt;
              &lt;reducer&gt;python reducer.py&lt;/reducer&gt;
          &lt;/streaming&gt;
          &lt;configuration&gt;
              &lt;property&gt;
                  &lt;name&gt;oozie.libpath&lt;/name&gt;
                  &lt;value&gt;${oozieLibPath}/mapreduce-streaming&lt;/value&gt;
              &lt;/property&gt;
              &lt;property&gt;
                  &lt;name&gt;mapred.input.dir&lt;/name&gt;
                  &lt;value&gt;${inputDir}&lt;/value&gt;
              &lt;/property&gt;
              &lt;property&gt;
                  &lt;name&gt;mapred.output.dir&lt;/name&gt;
                  &lt;value&gt;${outputDir}&lt;/value&gt;
              &lt;/property&gt;
              &lt;property&gt;
                  &lt;name&gt;mapred.reduce.tasks&lt;/name&gt;
                  &lt;value&gt;4&lt;/value&gt;
              &lt;/property&gt;
          &lt;/configuration&gt;
          &lt;file&gt;${appPath}/mapper.py#mapper.py&lt;/file&gt;
          &lt;file&gt;${appPath}/reducer.py#reducer.py&lt;/file&gt;
      &lt;/map-reduce&gt;
      &lt;ok to="sqoopAction"/&gt;
      &lt;error to="killJobAction"/&gt;
      &lt;/action&gt;
  
      &lt;action name="sqoopAction"&gt;
          &lt;sqoop xmlns="uri:oozie:sqoop-action:0.2"&gt;
              &lt;job-tracker&gt;${jobTracker}&lt;/job-tracker&gt;
              &lt;name-node&gt;${nameNode}&lt;/name-node&gt;
              &lt;command&gt;export --connect jdbc:mysql://localhost/database --username root --password myPwd --table tableName --direct --export-dir /user/hduser/oozieProject/workflow/output&lt;/command&gt;
          &lt;/sqoop&gt;        
      &lt;ok to="end"/&gt;
      &lt;error to="killJobAction"/&gt;
      &lt;/action&gt;
  &lt;kill name="killJobAction"&gt;
  &lt;message&gt;"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"&lt;/message&gt;
  &lt;/kill&gt;
  &lt;end name="end" /&gt;
  </code></pre>
  
  <p></p>
  
  <p>Please advice what could have gone wrong?</p>
  
  <p>Thank you</p>