title: regexp in PySpark
body: <p>I am trying to reproduce the results of the django ORM query in pyspark:</p>
  
  <pre><code>social_filter = '(facebook|flipboard|linkedin|pinterest|reddit|twitter)'
  Collection.objects.filter(social__iregex=social_filter)
  </code></pre>
  
  <p>My main problem is that it should be case insensitive.</p>
  
  <p>I have tried this:</p>
  
  <pre><code>social_filter = "social ILIKE 'facebook' OR social ILIKE 'flipboard' OR social ILIKE 'linkedin' OR social ILIKE 'pinterest' OR social ILIKE 'reddit' OR social ILIKE 'twitter'"
  df = sessions.filter(social_filter)
  </code></pre>
  
  <p>which result in the following error:</p>
  
  <pre><code>Py4JJavaError: An error occurred while calling o31.filter.
  : java.lang.RuntimeException: [1.22] failure: end of input expected
  
  social ILIKE 'facebook' OR social ILIKE 'flipboard' OR social ILIKE 'linkedin' OR social ILIKE 'pinterest' OR social ILIKE 'reddit' OR social ILIKE 'twitter'
  </code></pre>
  
  <p>And the following expression:</p>
  
  <pre><code>social_filter = "social  ~* (facebook|flipboard|linkedin|pinterest|reddit|twitter)"
  df = sessions.filter(social_filter)
  </code></pre>
  
  <p>crashes with this:</p>
  
  <pre><code>Py4JJavaError: An error occurred while calling o31.filter.
  : java.lang.RuntimeException: [1.17] failure: identifier expected
  
  social  ~* (facebook|flipboard|linkedin|pinterest|reddit|twitter)
         ^
      at scala.sys.package$.error(package.scala:27)
      at org.apache.spark.sql.catalyst.SqlParser.parseExpression(SqlParser.scala:45)
      at org.apache.spark.sql.DataFrame.filter(DataFrame.scala:652)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  </code></pre>
  
  <p>please, help!</p>