title: Using regular expression in spark
body: <p>I have trouble in using regular expression.
  My sample data is:</p>
  
  <pre><code>12 13 hello hiiii hhhhh
  
  this doesnt have numeric so should be removed
  Even this line should be excluded
  `12` this line contains numeric shouldn't exclude
  Hope even this line should be excluded
  
  scala&gt; val pattern = "[a-z][A-Z]".r                                                                                                                                                        
  </code></pre>
  
  <p>pattern: scala.util.matching.Regex = [a-z][A-Z]</p>
  
  <pre><code>scala&gt; val b = a.filter(line =&gt; !line.startsWith(pattern))
  &lt;console&gt;:31: error: type mismatch;
  </code></pre>
  
  <p>found   : scala.util.matching.Regex
   required: String
           val b = a.filter(line => !line.startsWith(pattern))
                                                     ^</p>
  
  <p>or if I use</p>
  
  <pre><code>scala&gt; val b = a.filter(line =&gt; !line.startsWith("[a-z][A-Z]".r)).take(3)                                                                                                                  
  </code></pre>
  
  <blockquote>
    <p>:29: error: type mismatch;<br>
    found   : scala.util.matching.Regex<br>
    required: String</p>
  </blockquote>
  
  <pre><code>     val b = a.filter(line =&gt; !line.startsWith("[a-z][A-Z]".r)).take(3)                                                                                                                                                            ^          
  </code></pre>
  
  <p>I am actually not sure how exactly to use regex in spark. Please help me out.</p>