title: How to make the first row as header when reading a file in PySpark and converting it to Pandas Dataframe
body: <p>I am reading a file in <code>PySpark</code> and forming the <code>rdd</code> of it. I then convert it to a normal <code>dataframe</code> and then to <code>pandas dataframe</code>. The issue that I am having is that there is header row in my input file and I want to make this as the header of dataframe columns as well but they are read in as an additional row and not as header. This is my current code:</p>
  
  <pre><code>def extract(line):
      return line
  
  
  input_file = sc.textFile('file1.txt').zipWithIndex().filter(lambda (line,rownum): rownum&gt;=0).map(lambda (line, rownum): line)
  
  input_data = (input_file
      .map(lambda line: line.split(";"))
      .filter(lambda line: len(line) &gt;=0 )
      .map(extract)) # Map to tuples
  
  df_normal = input_data.toDF()
  df= df_normal.toPandas()
  </code></pre>
  
  <p>Now when I look at the <code>df</code> then the header row of text file becomes the first row of <code>dataframe</code> and there is additional header in <code>df</code> with <code>0,1,2...</code> as header. How can I make the first row as header?</p>