title: PySpark - Remove first row from Dataframe
body: <p>I have a .txt file with a header, which I'd like to remove. The file looks like this:</p>
  
  <pre><code>Entry  Per  Account     Description               
   16524  01  3930621977  TXNPUES                     
  191675  01  2368183100  OUNHQEX            
  191667  01  3714468136  GHAKASC             
  191673  01  2632703881  PAHFSAP              
   80495  01  2766389794  XDZANTV                    
   80507  01  4609266335  BWWYEZL                   
   80509  01  1092717420  QJYPKVO                  
   80497  01  3386366766  SOQLCMU                  
  191669  01  5905893739  FYIWNKA             
  191671  01  2749355876  CBMJTLP 
  </code></pre>
  
  <pre class="lang-py prettyprint-override"><code># Create spark session
  spark = SparkSession.builder.master("local").appName("fixed-width"                          )\
                                              .config("spark.some.config.option", "some-value")\
                                              .getOrCreate()
  
  # Read in fixed-width text file into DataFrame
  df = spark.read.option("header"     , "true" )\
                 .option("inferSchema", "true" )\
                 .text(file                    )
  df.show()
  df.printSchema()
  </code></pre>
  
  <p>Which returns:</p>
  
  <pre><code>+--------------------+
  |               value|
  +--------------------+
  |Entry  Per  Accou...|
  | 16524  01  39306...|
  |191675  01  23681...|
  |191667  01  37144...|
  |191673  01  26327...|
  | 80495  01  27663...|
  | 80507  01  46092...|
  | 80509  01  10927...|
  | 80497  01  33863...|
  |191669  01  59058...|
  |191671  01  27493...|
  +--------------------+
  
  root
   |-- value: string (nullable = true)
  </code></pre>
  
  <p>I can grab the header:</p>
  
  <pre class="lang-py prettyprint-override"><code>header = df.first()
  header
  </code></pre>
  
  <p>which returns:</p>
  
  <pre><code>Row(value='Entry  Per  GL Account  Description               ')
  </code></pre>
  
  <p>and then split into distinct columns:</p>
  
  <pre class="lang-py prettyprint-override"><code># Take the fixed width file and split into 3 distinct columns
  sorted_df = df.select(
      df.value.substr( 1,  6).alias('Entry'      ),
      df.value.substr( 8,  3).alias('Per'        ),
      df.value.substr(12, 11).alias('GL Account' ),
      df.value.substr(24, 11).alias('Description'),
  )
  
  sorted_df.show()
  sorted_df.printSchema()
  </code></pre>
  
  <p>which returns:</p>
  
  <pre><code>+------+---+-----------+-----------+
  | Entry|Per| GL Account|Description|
  +------+---+-----------+-----------+
  |Entry |Per| GL Account| Descriptio|
  | 16524| 01| 3930621977| TXNPUES   |
  |191675| 01| 2368183100| OUNHQEX   |
  |191667| 01| 3714468136| GHAKASC   |
  |191673| 01| 2632703881| PAHFSAP   |
  | 80495| 01| 2766389794| XDZANTV   |
  | 80507| 01| 4609266335| BWWYEZL   |
  | 80509| 01| 1092717420| QJYPKVO   |
  | 80497| 01| 3386366766| SOQLCMU   |
  |191669| 01| 5905893739| FYIWNKA   |
  |191671| 01| 2749355876|   CBMJTLP |
  +------+---+-----------+-----------+
  </code></pre>
  
  <p>Now you see that the header still appears as the first line in my dataframe here. I'm unsure of how to remove it. </p>
  
  <p>.iloc is not available, and I often see this approach, but this only works on an RDD:</p>
  
  <pre class="lang-py prettyprint-override"><code>header = rdd.first()
  rdd.filter(lambda line: line != header)
  </code></pre>
  
  <p>So which alternatives are available?</p>