title: For SQL Server sqoop, import doesn't work when trying to pass the query from a file
body: <p>This is the script. <code>sqoop_sql.sh</code></p>
  
  <pre><code>query=$(cat ${SQL_SCRIPT})
  where_clause=" where dateadded &gt;= '2016-05-01' and dateadded &lt; '2016-06-01' and \$CONDITIONS"
  
  sqoop import -D mapreduce.job.queuename=s_sourcedata \
  --connect 'jdbc:sqlserver://connection' \
  --compression-codec org.apache.hadoop.io.compress.SnappyCodec \
  --username name \
  --password pas \
  --query "${query}${where_clause}" \
  --as-parquetfile \
  --split-by dateadded \
  --delete-target-dir \
  --target-dir prioritypass_history \
  -m 1
  </code></pre>
  
  <p>It doesn't work this way, but if I change first string to</p>
  
  <pre><code>query="select * FROM smth.[dbo].[tablename]"
  </code></pre>
  
  <p>it works.</p>
  
  <p>My action looks like this</p>
  
  <pre><code>&lt;action name="history" cred="hv_cred"&gt;
    &lt;shell xmlns="uri:oozie:shell-action:0.1"&gt;
      &lt;job-tracker&gt;${JOB_TRACKER}&lt;/job-tracker&gt;
      &lt;name-node&gt;${NAME_NODE}&lt;/name-node&gt;
      &lt;exec&gt;sqoop_sql.sh&lt;/exec&gt;
      &lt;env-var&gt;SQL_SCRIPT=${SQL_SCRIPT_HISTORY}&lt;/env-var&gt;
      ...
      &lt;file&gt;${WORKFLOW_APPLICATION_PATH}/bash/sqoop_sql.sh#sqoop_sql.sh&lt;/file&gt;
      &lt;file&gt;${WORKFLOW_APPLICATION_PATH}/oracle/${SQL_SCRIPT_HISTORY}#${SQL_SCRIPT_HISTORY}&lt;/file&gt;
    &lt;/shell&gt;
    &lt;ok to="end"/&gt;
    &lt;error to="kill"/&gt;
  &lt;/action&gt;
  </code></pre>
  
  <p>The thing is I used this same code to import data from oracle, changing only connection details. My only guess is that oozie doesn't like the fact the script is in folder <code>oracle</code>, but I'm not sure and don't know what to change it to if that's the case.</p>
  
  <p>PS</p>
  
  <p>I don't use sqoop action because there are some libs missing on the claster and it doesn't work.</p>
  
  <p>Edit</p>
  
  <p>I was wrong. The problem isn't in folder name.</p>
  
  <p>I left a bare shell action and shell script. After running it, it worked, then I killed the coordinator and restarted it. It didn't work. There were no changes to the code inbetween. Is it because of some settings, some error on claster? I don't know. Yet.</p>
  
  <pre><code>Log Type: stderr
  Log Upload Time: Tue Mar 20 18:52:54 +0300 2018
  Log Length: 240
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  
  Log Type: stdout
  Log Upload Time: Tue Mar 20 18:52:54 +0300 2018
  Log Length: 0
  
  Log Type: syslog
  Log Upload Time: Tue Mar 20 18:52:54 +0300 2018
  Log Length: 28581
  2018-03-20 18:52:33,683 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1518685780617_47486_000001
  2018-03-20 18:52:33,898 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
  2018-03-20 18:52:33,921 WARN [main] org.apache.hadoop.security.token.Token: Cannot find class for token kind HIVE_DELEGATION_TOKEN
  2018-03-20 18:52:33,921 WARN [main] org.apache.hadoop.security.token.Token: Cannot find class for token kind HIVE_DELEGATION_TOKEN
  Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 0a 6d 6d 69 6b 68 61 79 6c 6f 76 05 6f 6f 7a 69 65 3f 6f 6f 7a 69 65 2f 62 64 61 31 31 6e 6f 64 65 30 34 2e 6d 6f 73 63 6f 77 2e 61 6c 66 61 69 6e 74 72 61 2e 6e 65 74 40 42 44 41 2e 4d 4f 53 43 4f 57 2e 41 4c 46 41 49 4e 54 52 41 2e 4e 45 54 8a 01 62 44 1c a4 42 8a 01 62 68 29 28 42 8e 03 82 07
  2018-03-20 18:52:33,921 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@6bf0219d)
  2018-03-20 18:52:33,923 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN, Service: 172.25.55.232:8032,172.25.55.233:8032, Ident: (owner=user, renewer=yarn, realUser=oozie/host@host, issueDate=1521561150672, maxDate=1522165950672, sequenceNumber=82589, masterKeyId=554)
  2018-03-20 18:52:33,924 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:bda11, Ident: (HDFS_DELEGATION_TOKEN token 270591 for user)
  2018-03-20 18:52:33,924 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: MR_DELEGATION_TOKEN, Service: 172.25.55.232:10020, Ident: (owner=user, renewer=yarn, realUser=oozie/host@host, issueDate=1521561150796, maxDate=1522165950796, sequenceNumber=1524, masterKeyId=35)
  2018-03-20 18:52:34,414 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  2018-03-20 18:52:34,430 WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
  2018-03-20 18:52:34,489 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:user (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,490 WARN [main] org.apache.hadoop.ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,490 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:user (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,542 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
  2018-03-20 18:52:34,543 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
  2018-03-20 18:52:34,642 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
  2018-03-20 18:52:34,643 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
  2018-03-20 18:52:34,644 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
  2018-03-20 18:52:34,644 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
  2018-03-20 18:52:34,645 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
  2018-03-20 18:52:34,646 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
  2018-03-20 18:52:34,646 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
  2018-03-20 18:52:34,647 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
  2018-03-20 18:52:34,697 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://bda11:8020]
  2018-03-20 18:52:34,723 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://bda11:8020]
  2018-03-20 18:52:34,747 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://bda11:8020]
  2018-03-20 18:52:34,755 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:user (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,755 WARN [main] org.apache.hadoop.ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,755 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:user (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
  2018-03-20 18:52:34,763 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
  2018-03-20 18:52:34,805 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
  2018-03-20 18:52:34,983 WARN [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics system not started: org.apache.commons.configuration.ConfigurationException: Unable to load the configuration from the URL file:/var/run/cloudera-scm-agent/process/22786-yarn-NODEMANAGER/hadoop-metrics2.properties
  2018-03-20 18:52:35,035 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1518685780617_47486 to jobTokenSecretManager
  2018-03-20 18:52:35,133 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1518685780617_47486 because: not enabled;
  2018-03-20 18:52:35,156 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1518685780617_47486 = 0. Number of splits = 1
  2018-03-20 18:52:35,156 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1518685780617_47486 = 0
  2018-03-20 18:52:35,156 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1518685780617_47486Job Transitioned from NEW to INITED
  2018-03-20 18:52:35,157 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1518685780617_47486.
  2018-03-20 18:52:35,183 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
  2018-03-20 18:52:35,190 INFO [Socket Reader #1 for port 38335] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38335
  2018-03-20 18:52:35,204 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
  2018-03-20 18:52:35,229 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  2018-03-20 18:52:35,229 INFO [IPC Server listener on 38335] org.apache.hadoop.ipc.Server: IPC Server listener on 38335: starting
  2018-03-20 18:52:35,230 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at host/host:38335
  2018-03-20 18:52:35,288 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
  2018-03-20 18:52:35,292 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
  2018-03-20 18:52:35,297 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
  2018-03-20 18:52:35,306 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
  2018-03-20 18:52:35,331 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
  2018-03-20 18:52:35,331 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
  2018-03-20 18:52:35,334 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
  2018-03-20 18:52:35,334 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
  2018-03-20 18:52:35,342 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 24744
  2018-03-20 18:52:35,342 INFO [main] org.mortbay.log: jetty-6.1.26.cloudera.4
  2018-03-20 18:52:35,370 INFO [main] org.mortbay.log: Extract jar:file:/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/jars/hadoop-yarn-common-2.6.0-cdh5.7.0.jar!/webapps/mapreduce to ./tmp/Jetty_0_0_0_0_24744_mapreduce____z2u350/webapp
  2018-03-20 18:52:35,578 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:24744
  2018-03-20 18:52:35,578 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 24744
  2018-03-20 18:52:35,859 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
  2018-03-20 18:52:35,864 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
  2018-03-20 18:52:35,865 INFO [Socket Reader #1 for port 27207] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 27207
  2018-03-20 18:52:35,897 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  2018-03-20 18:52:35,897 INFO [IPC Server listener on 27207] org.apache.hadoop.ipc.Server: IPC Server listener on 27207: starting
  2018-03-20 18:52:35,918 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
  2018-03-20 18:52:35,918 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
  2018-03-20 18:52:35,918 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
  2018-03-20 18:52:35,986 INFO [main] org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider: Failing over to rm340
  2018-03-20 18:52:36,010 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: &lt;memory:29304, vCores:19&gt;
  2018-03-20 18:52:36,010 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.s_sourcedata
  2018-03-20 18:52:36,014 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
  2018-03-20 18:52:36,014 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
  2018-03-20 18:52:36,016 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
  2018-03-20 18:52:36,023 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1518685780617_47486Job Transitioned from INITED to SETUP
  2018-03-20 18:52:36,024 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
  2018-03-20 18:52:36,027 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
  2018-03-20 18:52:36,034 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1518685780617_47486Job Transitioned from SETUP to RUNNING
  2018-03-20 18:52:36,064 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1518685780617_47486_m_000000 Task Transitioned from NEW to SCHEDULED
  2018-03-20 18:52:36,066 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
  2018-03-20 18:52:36,067 INFO [Thread-54] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:&lt;memory:2048, vCores:1&gt;
  2018-03-20 18:52:36,090 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1518685780617_47486, File: hdfs://bda11:8020/user/user/.staging/job_1518685780617_47486/job_1518685780617_47486_1.jhist
  2018-03-20 18:52:36,298 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://bda11:8020]
  2018-03-20 18:52:37,013 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
  2018-03-20 18:52:37,047 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1518685780617_47486: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=&lt;memory:91136, vCores:111&gt; knownNMs=6
  2018-03-20 18:52:38,055 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
  2018-03-20 18:52:38,089 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e16_1518685780617_47486_01_000002 to attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:38,090 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
  2018-03-20 18:52:38,136 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Job jar is not present. Not adding any jar to the list of resources.
  2018-03-20 18:52:38,204 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /user/user/.staging/job_1518685780617_47486/job.xml
  2018-03-20 18:52:38,212 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #4 tokens and #1 secret keys for NM use for launching container
  2018-03-20 18:52:38,212 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 5
  2018-03-20 18:52:38,212 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
  2018-03-20 18:52:38,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
  2018-03-20 18:52:38,248 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e16_1518685780617_47486_01_000002 taskAttempt attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:38,249 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:38,250 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : host:8041
  2018-03-20 18:52:38,297 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1518685780617_47486_m_000000_0 : 13562
  2018-03-20 18:52:38,299 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1518685780617_47486_m_000000_0] using containerId: [container_e16_1518685780617_47486_01_000002 on NM: [host:8041]
  2018-03-20 18:52:38,302 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
  2018-03-20 18:52:38,302 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1518685780617_47486_m_000000 Task Transitioned from SCHEDULED to RUNNING
  2018-03-20 18:52:39,092 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1518685780617_47486: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=&lt;memory:89088, vCores:110&gt; knownNMs=6
  2018-03-20 18:52:41,461 INFO [Socket Reader #1 for port 27207] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1518685780617_47486 (auth:SIMPLE)
  2018-03-20 18:52:41,476 INFO [Socket Reader #1 for port 27207] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1518685780617_47486 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
  2018-03-20 18:52:41,483 INFO [IPC Server handler 0 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1518685780617_47486_m_17592186044418 asked for a task
  2018-03-20 18:52:41,483 INFO [IPC Server handler 0 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1518685780617_47486_m_17592186044418 given task: attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,454 INFO [IPC Server handler 5 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1518685780617_47486_m_000000_0 is : 0.0
  2018-03-20 18:52:46,529 INFO [IPC Server handler 4 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,529 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
  2018-03-20 18:52:46,530 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1518685780617_47486_m_000000_0 given a go for committing the task output.
  2018-03-20 18:52:46,530 INFO [IPC Server handler 1 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,531 INFO [IPC Server handler 1 on 27207] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1518685780617_47486_m_000000_0:true
  2018-03-20 18:52:46,593 INFO [IPC Server handler 3 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1518685780617_47486_m_000000_0 is : 1.0
  2018-03-20 18:52:46,599 INFO [IPC Server handler 2 on 27207] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,601 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
  2018-03-20 18:52:46,609 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,610 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1518685780617_47486_m_000000 Task Transitioned from RUNNING to SUCCEEDED
  2018-03-20 18:52:46,611 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
  2018-03-20 18:52:46,612 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1518685780617_47486Job Transitioned from RUNNING to COMMITTING
  2018-03-20 18:52:46,612 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
  2018-03-20 18:52:46,641 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 
  2018-03-20 18:52:46,642 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1518685780617_47486Job Transitioned from COMMITTING to SUCCEEDED
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: true
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
  2018-03-20 18:52:46,643 INFO [Thread-73] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
  2018-03-20 18:52:46,668 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://bda11:8020/user/user/.staging/job_1518685780617_47486/job_1518685780617_47486_1.jhist to hdfs://bda11:8020/user/history/done_intermediate/user/job_1518685780617_47486-1521561150808-user-oozie%3Alauncher%3AT%3Dshell%3AW%3Dserver_import%3AA%3Dhistory%3AI-1521561166640-1-0-SUCCEEDED-root.s_sourcedata-1521561156018.jhist_tmp
  2018-03-20 18:52:46,705 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://bda11:8020/user/history/done_intermediate/user/job_1518685780617_47486-1521561150808-user-oozie%3Alauncher%3AT%3Dshell%3AW%3Dserver_import%3AA%3Dhistory%3AI-1521561166640-1-0-SUCCEEDED-root.s_sourcedata-1521561156018.jhist_tmp to hdfs://bda11:8020/user/history/done_intermediate/user/job_1518685780617_47486-1521561150808-user-oozie%3Alauncher%3AT%3Dshell%3AW%3Dserver_import%3AA%3Dhistory%3AI-1521561166640-1-0-SUCCEEDED-root.s_sourcedata-1521561156018.jhist
  2018-03-20 18:52:46,706 INFO [Thread-73] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
  2018-03-20 18:52:46,706 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1518685780617_47486_m_000000_0
  2018-03-20 18:52:46,706 INFO [Thread-73] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : host:8041
  2018-03-20 18:52:46,716 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1518685780617_47486_m_000000_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED
  2018-03-20 18:52:46,717 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to 
  2018-03-20 18:52:46,718 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is http://host:19888/jobhistory/job/job_1518685780617_47486
  2018-03-20 18:52:46,722 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for application to be successfully unregistered.
  2018-03-20 18:52:47,723 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0
  2018-03-20 18:52:47,724 INFO [Thread-73] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://bda11 /user/user/.staging/job_1518685780617_47486
  2018-03-20 18:52:47,727 INFO [Thread-73] org.apache.hadoop.ipc.Server: Stopping server on 27207
  2018-03-20 18:52:47,727 INFO [IPC Server listener on 27207] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 27207
  2018-03-20 18:52:47,728 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  2018-03-20 18:52:47,728 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
  2018-03-20 18:52:47,729 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted
  </code></pre>