title: Hive SQL functions getting stuck during execution
body: <p>I am running Hive 1.2.1 and Hadoop 2.6.0 in a single node distribution. I have a simple JSON file in HDFS directory <code>/usr/&lt;name&gt;/test.json</code>. </p>
  
  <p>I create a HIVE table and load this data using following script: </p>
  
  <pre><code>CREATE EXTERNAL TABLE json_table (json string) LOCATION '/user/&lt;name&gt;/test.json' 
  </code></pre>
  
  <p>This happens successfully. Now I am querying the table from command line:</p>
  
  <pre><code>hive&gt; select get_json_object(json_table.json,'$.doc') from json_table;
  
  OK
  123
  456
  789
  345
  987
  </code></pre>
  
  <p>Now, the problem part:</p>
  
  <pre><code>select count(*) from json_table;
  </code></pre>
  
  <p>This query initiates MR jobs behind the scenes but never finishes. I have to manually kill the query from command line. I can't infer much from the logs or from the console, when I enable them from command line.</p>
  
  <p>In short any queries that have SQL functions, the query gets stuck in Hive!! Am I missing any jars needed for SQL functions, which need to be put in HDFS?</p>
  
  <p>Pasting the stack trace from command line:</p>
  
  <pre><code>hive&gt; select count(*) from json_table;
  15/11/28 21:44:22 [main]: INFO log.PerfLogger: &lt;PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:22 [main]: INFO log.PerfLogger: &lt;PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:22 [main]: INFO log.PerfLogger: &lt;PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:22 [main]: INFO log.PerfLogger: &lt;PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:22 [main]: INFO parse.ParseDriver: Parsing command: select count(*) from json_table
  15/11/28 21:44:23 [main]: INFO parse.ParseDriver: Parse Completed   
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=parse start=1448765062471 end=1448765063122 duration=651 from=org.apache.hadoop.hive.ql.Driver&gt;    
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver&gt;    
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Starting Semantic Analysis    
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Completed phase 1 of Semantic Analysis    
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Get metadata for source tables    
  15/11/28 21:44:23 [main]: INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=json_table    
  15/11/28 21:44:23 [main]: INFO HiveMetaStore.audit: ugi=sriramvaradharajan  ip=unknown-ip-addr  cmd=get_table : db=default tbl=json_table       
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Get metadata for subqueries    
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Get metadata for destination tables    
  15/11/28 21:44:23 [main]: INFO ql.Context: New scratch dir is hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1    
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis    
  15/11/28 21:44:23 [main]: INFO parse.BaseSemanticAnalyzer: Not invoking CBO because the statement has too few joins    
  15/11/28 21:44:23 [main]: INFO common.FileUtils: Creating directory if it doesn't exist: hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1/-mr-10000/.hive-staging_hive_2015-11-28_21-44-22_469_6217205930037435432-1
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Set stats collection dir : hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1/-mr-10000/.hive-staging_hive_2015-11-28_21-44-22_469_6217205930037435432-1/-ext-10002    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for FS(6)    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for SEL(5)    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for GBY(4)    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for RS(3)    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for GBY(2)    
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for SEL(1)
  15/11/28 21:44:23 [main]: INFO ppd.OpProcFactory: Processing for TS(0)    
  15/11/28 21:44:23 [main]: INFO optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0]}    
  15/11/28 21:44:23 [main]: INFO optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0]}    
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=partition-retrieving from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner&gt;    
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=partition-retrieving start=1448765063731 end=1448765063732 duration=1 from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner&gt;    
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable    
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Found 0 null table scans    
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Found 0 null table scans
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable
  15/11/28 21:44:23 [main]: INFO physical.NullScanTaskDispatcher: Found 0 null table scans
  15/11/28 21:44:23 [main]: INFO parse.CalcitePlanner: Completed plan generation
  15/11/28 21:44:23 [main]: INFO ql.Driver: Semantic Analysis Completed
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=semanticAnalyze start=1448765063125 end=1448765063749 duration=624 from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:23 [main]: INFO exec.ListSinkOperator: Initializing operator OP[7]
  15/11/28 21:44:23 [main]: INFO exec.ListSinkOperator: Initialization Done 7 OP
  15/11/28 21:44:23 [main]: INFO exec.ListSinkOperator: Operator 7 OP initialized
  15/11/28 21:44:23 [main]: INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=compile start=1448765062445 end=1448765063772 duration=1327 from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:23 [main]: INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:23 [main]: INFO ql.Driver: Starting command(queryId=sriramvaradharajan_20151128214422_cd5ef5dc-1862-4aeb-a4d1-361dc7be0056): select count(*) from json_table
  Query ID = sriramvaradharajan_20151128214422_cd5ef5dc-1862-4aeb-a4d1-361dc7be0056
  15/11/28 21:44:23 [main]: INFO ql.Driver: Query ID = sriramvaradharajan_20151128214422_cd5ef5dc-1862-4aeb-a4d1-361dc7be0056
  Total jobs = 1
  15/11/28 21:44:23 [main]: INFO ql.Driver: Total jobs = 1
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=TimeToSubmit start=1448765062445 end=1448765063775 duration=1330 from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver&gt;
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=task.MAPRED.Stage-1 from=org.apache.hadoop.hive.ql.Driver&gt;
  Launching Job 1 out of 1
  15/11/28 21:44:23 [main]: INFO ql.Driver: Launching Job 1 out of 1
  15/11/28 21:44:23 [main]: INFO ql.Driver: Starting task [Stage-1:MAPRED] in serial mode
  Number of reduce tasks determined at compile time: 1
  15/11/28 21:44:23 [main]: INFO exec.Task: Number of reduce tasks determined at compile time: 1
  In order to change the average load for a reducer (in bytes):
  15/11/28 21:44:23 [main]: INFO exec.Task: In order to change the average load for a reducer (in bytes):
    set hive.exec.reducers.bytes.per.reducer=&lt;number&gt;
  15/11/28 21:44:23 [main]: INFO exec.Task:   set hive.exec.reducers.bytes.per.reducer=&lt;number&gt;
  In order to limit the maximum number of reducers:
  15/11/28 21:44:23 [main]: INFO exec.Task: In order to limit the maximum number of reducers:
    set hive.exec.reducers.max=&lt;number&gt;
  15/11/28 21:44:23 [main]: INFO exec.Task:   set hive.exec.reducers.max=&lt;number&gt;
  In order to set a constant number of reducers:
  15/11/28 21:44:23 [main]: INFO exec.Task: In order to set a constant number of reducers:
    set mapreduce.job.reduces=&lt;number&gt;
  15/11/28 21:44:23 [main]: INFO exec.Task:   set mapreduce.job.reduces=&lt;number&gt;
  15/11/28 21:44:23 [main]: INFO ql.Context: New scratch dir is hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1
  15/11/28 21:44:23 [main]: INFO mr.ExecDriver: Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
  15/11/28 21:44:23 [main]: INFO exec.Utilities: Processing alias json_table
  15/11/28 21:44:23 [main]: INFO exec.Utilities: Adding input file hdfs://localhost:9000/user/sriramvaradharajan
  15/11/28 21:44:23 [main]: INFO exec.Utilities: Content Summary not cached for hdfs://localhost:9000/user/sriramvaradharajan
  15/11/28 21:44:23 [main]: INFO ql.Context: New scratch dir is hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities&gt;
  15/11/28 21:44:23 [main]: INFO exec.Utilities: Serializing MapWork via kryo
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=serializePlan start=1448765063830 end=1448765063973 duration=143 from=org.apache.hadoop.hive.ql.exec.Utilities&gt;
  15/11/28 21:44:23 [main]: INFO Configuration.deprecation: mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities&gt;
  15/11/28 21:44:23 [main]: INFO exec.Utilities: Serializing ReduceWork via kryo
  15/11/28 21:44:23 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=serializePlan start=1448765063980 end=1448765063994 duration=14 from=org.apache.hadoop.hive.ql.exec.Utilities&gt;
  15/11/28 21:44:23 [main]: ERROR mr.ExecDriver: yarn
  15/11/28 21:44:24 [main]: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  15/11/28 21:44:24 [main]: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  15/11/28 21:44:24 [main]: INFO exec.Utilities: PLAN PATH = hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1/-mr-10004/be3b1969-bd83-4db2-b51c-c5e7b6596f79/map.xml
  15/11/28 21:44:24 [main]: INFO exec.Utilities: PLAN PATH = hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1/-mr-10004/be3b1969-bd83-4db2-b51c-c5e7b6596f79/reduce.xml
  15/11/28 21:44:24 [main]: WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
  15/11/28 21:44:24 [main]: INFO log.PerfLogger: &lt;PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat&gt;
  15/11/28 21:44:24 [main]: INFO exec.Utilities: PLAN PATH = hdfs://localhost:9000/tmp/hive/sriramvaradharajan/2c73a21d-3600-4325-8f71-811db3f5368d/hive_2015-11-28_21-44-22_469_6217205930037435432-1/-mr-10004/be3b1969-bd83-4db2-b51c-c5e7b6596f79/map.xml
  15/11/28 21:44:24 [main]: INFO io.CombineHiveInputFormat: Total number of paths: 1, launching 1 threads to check non-combinable ones.
  15/11/28 21:44:24 [main]: INFO io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for hdfs://localhost:9000/user/sriramvaradharajan; using filter path hdfs://localhost:9000/user/sriramvaradharajan
  15/11/28 21:44:24 [main]: INFO input.FileInputFormat: Total input paths to process : 2
  15/11/28 21:44:24 [main]: INFO input.CombineFileInputFormat: DEBUG: Terminated node allocation with : CompletedNodes: 1, size left: 0
  15/11/28 21:44:24 [main]: INFO io.CombineHiveInputFormat: number of splits 1
  15/11/28 21:44:24 [main]: INFO io.CombineHiveInputFormat: Number of all splits 1
  15/11/28 21:44:24 [main]: INFO log.PerfLogger: &lt;/PERFLOG method=getSplits start=1448765064446 end=1448765064479 duration=33 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat&gt;
  15/11/28 21:44:24 [main]: INFO mapreduce.JobSubmitter: number of splits:1
  15/11/28 21:44:24 [main]: INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1448752941502_0008
  15/11/28 21:44:24 [main]: INFO impl.YarnClientImpl: Submitted application application_1448752941502_0008    
  15/11/28 21:44:24 [main]: INFO mapreduce.Job: The url to track the job: h--ttp://admins-MacBook-Pro.local:8088/proxy/application_1448752941502_0008/
  Starting Job = job_1448752941502_0008, Tracking URL = h--ttp://admins-MacBook-Pro.local:8088/proxy/application_1448752941502_0008/
  15/11/28 21:44:24 [main]: INFO exec.Task: Starting Job = job_1448752941502_0008, Tracking URL = h--ttp://admins-MacBook-Pro.local:8088/proxy/application_1448752941502_0008/
  Kill Command = /usr/local/Cellar/hadoop/2.6.0/bin/hadoop job  -kill job_1448752941502_0008
  15/11/28 21:44:24 [main]: INFO exec.Task: Kill Command = /usr/local/Cellar/hadoop/2.6.0/bin/hadoop job  -kill job_1448752941502_0008
  </code></pre>