title: PySpark - Using lists inside LIKE operator
body: <p>I would like to use list inside the LIKE operator on pyspark in order to create a column.</p>
  
  <p>I have the following input df :</p>
  
  <p><strong><code>input_df :</code></strong></p>
  
  <pre><code>+------+--------------------+-------+
  |    ID|           customers|country|
  +------+--------------------+-------+
  |161   |xyz Limited         |U.K.   |
  |262   |ABC  Limited        |U.K.   |
  |165   |Sons &amp; Sons         |U.K.   |
  |361   |TÜV GmbH            |Germany|
  |462   |Mueller GmbH        |Germany|
  |369   |Schneider AG        |Germany|
  |467   |Sahm UG             |Austria|
  +------+--------------------+-------+
  </code></pre>
  
  <p>I would like to add a column CAT_ID. CAT_ID takes value 1 if "ID" contains "16" or "26". CAT_ID takes value 2 if "ID" contains "36" or "46".<br>
  So, I want my output df to look like this -</p>
  
  <p><strong><code>The desired output_df :</code></strong></p>
  
  <pre><code>+------+--------------------+-------+-------+
  |    ID|           customers|country|Cat_ID |
  +------+--------------------+-------+-------+
  |161   |xyz Limited         |U.K.   |1      |
  |262   |ABC  Limited        |U.K.   |1      |
  |165   |Sons &amp; Sons         |U.K.   |1      |
  |361   |TÜV GmbH            |Germany|2      |
  |462   |Mueller GmbH        |Germany|2      |
  |369   |Schneider AG        |Germany|2      |
  |467   |Sahm UG             |Austria|2      |
  +------+--------------------+-------+-------+
  </code></pre>
  
  <p>I am interested in learning how this can be done using LIKE statement and lists.</p>
  
  <p>I know how to implement it without list, which works perfectly:</p>
  
  <pre><code>from pyspark.sql import functions as F
  
  def add_CAT_ID(df):
      return df.withColumn(
          'CAT_ID', 
          F.when( ( (F.col('ID').like('16%')) | (F.col('ID').like('26%'))  ) , "1") \
           .when( ( (F.col('ID').like('36%')) | (F.col('ID').like('46%'))  ) , "2") \
           .otherwise('999')
      )
  
  
      output_df = add_CAT_ID(input_df)
  </code></pre>
  
  <p>However, I would love to use list and have something like:</p>
  
  <pre><code>list1 =['16', '26']
  list2 =['36', '46']
  
  
  def add_CAT_ID(df):
      return df.withColumn(
          'CAT_ID', 
          F.when( ( (F.col('ID').like(list1 %))  ) , "1") \
           .when( ( (F.col('ID').like('list2 %'))  ) , "2") \
           .otherwise('999')
      )
  
  
      output_df = add_CAT_ID(input_df)
  </code></pre>
  
  <p>Thanks a lot in advance,</p>