title: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 21
body: <p>I have yarn cluster with spark(1.6.1), hdfs and hive(2.1). My workflows worked fine for few months till this day (without any changes in code / on environments). I started to get errors like this:</p>
  
  <pre><code>org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 21
  Serialization trace:
  outputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc)
  aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork)
  invertedWorkGraph (org.apache.hadoop.hive.ql.plan.SparkWork)
      at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:238)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultSerializers$ClassSerializer.read(DefaultSerializers.java:226)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:745)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:113)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:131)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
      at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
      at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
      at org.apache.hadoop.hive.ql.exec.spark.KryoSerializer.deserialize(KryoSerializer.java:49)
      at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient$JobStatusJob.call(RemoteHiveSparkClient.java:318)
      at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:366)
      at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:335)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
  </code></pre>
  
  <p>Using hive i can do simple selects, but every other operation which needs spark ends with <code>Error: Error while processing statement: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask (state=08S01,code=3)</code> in console, and error above in yarn logs.
  Now my every hive database is paralyzed (i have few). I was trying to solve this problem whole day, but couldnt do antything (hive restart, yarn node's restarts, changing yarn master).</p>
  
  <p>What do you think causes the problem and how can it be solved?</p>