title: How to use multiple regex patterns using rlike in pyspark
body: <p>I have to use multiple patterns to filter a large file. The problem is I am not sure about the efficient way of applying multiple patterns using <code>rlike</code>. As an example</p>
  
  <pre class="lang-python prettyprint-override"><code>df = spark.createDataFrame(
      [
          ('www 17 north gate',),
          ('aaa 45 north gate',),
          ('bbb 56 west gate',),
          ('ccc 56 south gate',),
          ('Michigan gate',),
          ('Statue of Liberty',),
          ('57 adam street',),
          ('19 west main street',),
          ('street burger',)
      ],
      [ 'poi']
  )
  
  df.show()
  +-------------------+
  |                poi|
  +-------------------+
  |  www 17 north gate|
  |  aaa 45 north gate|
  |   bbb 56 west gate|
  |  ccc 56 south gate|
  |      Michigan gate|
  |  Statue of Liberty|
  |     57 adam street|
  |19 west main street|
  |      street burger|
  +-------------------+
  </code></pre>
  
  <p>If i use the following two patterns from the data I could do </p>
  
  <pre class="lang-python prettyprint-override"><code>pat1="(aaa|bbb|ccc) [0-9]+ (north|south|west|east) gate$"
  pat2="[0-9]+ [a-z\s]+ street$"
  df.filter(~df['poi'].rlike(pat2)).filter(~df['poi'].rlike(pat1)).show()
  +-----------------+
  |              poi|
  +-----------------+
  |www 45 north gate|
  |    Michigan gate|
  |Statue of Liberty|
  |    street burger|
  +-----------------+
  </code></pre>
  
  <p>how about if I have 40 different patterns? I guess I can use a loop like this  </p>
  
  <pre class="lang-python prettyprint-override"><code>for pat in [pat1,pat2,....,patn]:
      df = df.filter(~df['poi'].rlike(pat))
  </code></pre>
  
  <p>Is this the right approach? The original data is in Chinese so please ignore if the patterns are efficient or not. I am just interested to see how I handle multiple regex patterns.</p>