title: How to create a spark DataFrame from sequenceFile
body: <p>I am working with spark 1.5. I want to create a <code>dataframe</code> from files in HDFS. The HDFS files contain <code>json</code> data with a large number of fields in sequence input file format.</p>
  
  <p>Is there a way to do this elegantly in java? I don't know the structure/fields of json in advance.</p>
  
  <p>I am able to take input from sequence file as RDD as follows:</p>
  
  <pre><code>JavaPairRDD&lt;LongWritable,BytesWritable&gt; inputRDD = jsc.sequenceFile("s3n://key_id:secret_key@file/path", LongWritable.class, BytesWritable.class);
  JavaRDD&lt;String&gt; events = inputRDD.map(
      new Function&lt;Tuple2&lt;LongWritable,BytesWritable&gt;, String&gt;() {
          public String call(Tuple2&lt;LongWritable,BytesWritable&gt; tuple) {
              return Text.decode(tuple._2.getBytes());
          }
      }
  );
  </code></pre>
  
  <p>How can I create a data frame from this RDD?</p>