title: How to filter from RDDs and DataFrames in Spark?
body: <p>I have a <code>.tsv</code> file <code>pageviews_by_second</code> consisting of the <code>timestamp</code> <code>site</code> and <code>requests</code>fields:</p>
  
  <pre><code>"timestamp"              "site"   "requests"
  "2015-03-16T00:09:55"   "mobile"    1595
  "2015-03-16T00:10:39"   "mobile"    1544
  "2015-03-16T00:19:39"   "desktop"   2460
  </code></pre>
  
  <p>I want the first row to be gone, because it leads to errors in the operations I have to perform on the data.</p>
  
  <p>I tried doing it in the following ways:</p>
  
  <p>1.Filtering the RDD before splitting it  </p>
  
  <pre><code>val RDD1 = sc.textFile("pageviews_by_second")       
  val top_row = RDD1.first() 
  //returns: top_row: String = "timestamp"    "site"  "requests"    
  val RDD2 = RDD1.filter(x =&gt; x!= top_row)
  RDD2.first() 
  //returns: "2015-03-16T00:09:55"    "mobile"    1595
  </code></pre>
  
  <p>2.Filtering the RDD after splitting it</p>
  
  <pre><code>val RDD1 = sc.textFile("pageviews_by_second").map(_.split("\t")
  RDD1.first()  //returns res0: Array[String] = Array("timestamp, 'site", "requests")
  val top_row = RDD1.first()
  val RDD2 = RDD1.filter(x =&gt; x!= top_row)
  RDD2.first() //returns: res1: Array[String] = Array("timestamp", "site" ,"requests")
  val RDD2 = RDD1.filter(x =&gt; x(0)!="timestamp" &amp;&amp; x(1)!="site" &amp;&amp; x(2)!="requests")
   RDD2.first() //returns: res1: Array[String] = Array("timestamp", "site" ,"requests")
  </code></pre>
  
  <p>3.Converting into a DataFrame using 'case class' and the filtering it</p>
  
  <pre><code>case class Wiki(timestamp: String, site: String, requests: String)
  val DF = sc.textFile("pageviews_by_second").map(_.split("\t")).map(w =&gt; Wiki(w(0), w(1), w(2))).toDF()
  val top_row = DF.first()
  //returns: top_row: org.apache.spark.sql.Row = ["timestamp","site","requests"]
  DF.filter(_ =&gt; _ != top_row)
  //returns: error: missing parameter type
  val DF2 = DF.filter(_ =&gt; _ != top_row2)
  </code></pre>
  
  <p>Why is only the 1st method able to filter out the first row while the other two aren't ? In method 3, why do I get the error and how can I rectify it ?</p>