title: How to retrieve Avro data from HDFS?
body: <p>I've created a JSON data, and an Avro schema for it:</p>
  
  <pre><code>{"username":"miguno","tweet":"Rock: Nerf paper, scissors is fine.","timestamp": 1366150681 }
  {"username":"BlizzardCS","tweet":"Works as intended.  Terran is IMBA.","timestamp": 1366154481 }
  </code></pre>
  
  <p>and</p>
  
  <blockquote>
    <blockquote>
      <p>{   "type" : "record",   "name" : "twitter_schema",   "namespace" : "com.miguno.avro",   "fields" : [ {
          "name" : "username",
          "type" : "string",
          "doc"  : "Name of the user account on Twitter.com"   }, {
          "name" : "tweet",
          "type" : "string",
          "doc"  : "The content of the user's Twitter message"   }, {
          "name" : "timestamp",
          "type" : "long",
          "doc"  : "Unix epoch time in seconds"   } ],   "doc:" : "A basic schema for storing Twitter messages" }</p>
    </blockquote>
  </blockquote>
  
  <p>I then converted it to Avro as follows:</p>
  
  <pre><code>java -jar ~/avro-tools-1.7.4.jar fromjson --schema-file twitter.avsc twitter.json &gt; twitter.avro
  </code></pre>
  
  <p>Put the file on hdfs with this:</p>
  
  <pre><code>hadoop fs -copyFromLocal twitter.avro &lt;path&gt;
  </code></pre>
  
  <p>And then In Spark CLI I issued the code:</p>
  
  <pre><code>import org.apache.avro.generic.GenericRecord
  import org.apache.avro.mapred.{AvroInputFormat, AvroWrapper}
  import org.apache.hadoop.io.NullWritable
  
  val path = "hdfs:///path/to/your/avro/folder"
  val avroRDD = sc.hadoopFile[AvroWrapper[GenericRecord], NullWritable, AvroInputFormat[GenericRecord]](path)
  </code></pre>
  
  <p>However when doing:</p>
  
  <pre><code>avroRDD.first
  </code></pre>
  
  <p>I'm facing the following exception:</p>
  
  <blockquote>
    <p>org.apache.spark.SparkException: Job aborted due to stage failure:
    Task 2.0 in stage 7.0 (TID 13) had a not serializable result:
    org.apache.avro.mapred.AvroWrapper    at
    org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185)
      at
    org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174)
      at
    org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173)
      at
    scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)</p>
  </blockquote>
  
  <p>What is a solution to it?</p>