title: TaskID.<init>(Lorg/apache/hadoop/mapreduce/JobID;Lorg/apache/hadoop/mapreduce/TaskType;I)V
body: <pre><code>val jobConf = new JobConf(hbaseConf)  
  jobConf.setOutputFormat(classOf[TableOutputFormat])  
  jobConf.set(TableOutputFormat.OUTPUT_TABLE, tablename)  
  
  val indataRDD = sc.makeRDD(Array("1,jack,15","2,Lily,16","3,mike,16"))  
  
  indataRDD.map(_.split(','))   
  val rdd = indataRDD.map(_.split(',')).map{arr=&gt;{  
  val put = new Put(Bytes.toBytes(arr(0).toInt))  
  put.add(Bytes.toBytes("cf"),Bytes.toBytes("name"),Bytes.toBytes(arr(1)))  
  put.add(Bytes.toBytes("cf"),Bytes.toBytes("age"),Bytes.toBytes(arr(2).toInt))  
  (new ImmutableBytesWritable, put)   
  }}  
    rdd.saveAsHadoopDataset(jobConf)  
  </code></pre>
  
  <p>When I run hadoop or spark jobs, I often meet the error:</p>
  
  <pre><code>Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID.&lt;init&gt;(Lorg/apache/hadoop/mapreduce/JobID;Lorg/apache/hadoop/mapreduce/TaskType;I)V
  at org.apache.spark.SparkHadoopWriter.setIDs(SparkHadoopWriter.scala:158)
  at org.apache.spark.SparkHadoopWriter.preSetup(SparkHadoopWriter.scala:60)
  at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1188)
  at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply(PairRDDFunctions.scala:1161)
  at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1.apply(PairRDDFunctions.scala:1161)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
  at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopDataset(PairRDDFunctions.scala:1161)
  at com.iteblog.App$.main(App.scala:62)
  at com.iteblog.App.main(App.scala)`
  </code></pre>
  
  <p>At the begin, I think, is the jar conflict, but I carefully checked the jar: there are no other jars. The spark and hadoop versions are:</p>
  
  <pre><code>&lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
  &lt;artifactId&gt;spark-core_2.11&lt;/artifactId&gt;
  &lt;version&gt;2.0.1&lt;/version&gt;`
  
  &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
  &lt;artifactId&gt;hadoop-core&lt;/artifactId&gt;
  &lt;version&gt;2.6.0-mr1-cdh5.5.0&lt;/version&gt;
  </code></pre>
  
  <p>And I found that the TaskID and TaskType are all in the <strong>hadoop-core</strong> jar, but not in the same package. Why the mapred.TaskID can refer the mapreduce.TaskType ?  </p>