title: Pattern matching with regular expression in spark dataframes using spark-shell
body: <p>Suppose we are given dataset ("DATA") like :</p>
  
  <pre><code>YEAR | FIRST NAME | LAST NAME | VARIABLES
  2008 | JOY        | ANDERSON  | spark|python|scala; 45;w/o sports;w datascience
  2008 | STEVEN     | JOHNSON   | Spark|R; 90|56
  2006 | NIHA       | DIVA      | w/o sports
  </code></pre>
  
  <p>and we have another dataset ("RESULT") like : </p>
  
  <pre><code>YEAR | FIRST NAME | LAST NAME 
  1992 | EMMA       | CENA 
  2008 | JOY        | ANDERSON
  2008 | STEVEN     | ANDERSON
  2006 | NIHA       | DIVA
  and so on.
  </code></pre>
  
  <p>The output should be ("RESULT") : </p>
  
  <pre><code>YEAR | FIRST NAME | LAST NAME | SUBJECT | SCORE | SPORTS | DATASCIENCE
  1992 | EMMA       | CENA      |         |       |        |              
  2008 | JOY        | ANDERSON  | SPARK   | 45    | FALSE  | TRUE
  2008 | JOY        | ANDERSON  | PYTHON  | 45    | FALSE  | TRUE
  2008 | JOY        | ANDERSON  | SCALA   | 45    | FALSE  | TRUE
  2008 | STEVEN     | ANDERSON  |         |       |        | 
  2006 | NIHA       | DIVA      |         |       | FALSE  | 
  2008 | STEVEN     | JOHNSON   | SPARK   | 90    |        |
  2008 | STEVEN     | JOHNSON   | SPARK   | 56    |        |
  2008 | STEVEN     | JOHNSON   | R       | 90    |        |
  2008 | STEVEN     | JOHNSON   | R       | 56    |        |
  and so on. 
  </code></pre>
  
  <p>Please note that there are some rows in DATA which are not present in RESULT and vice-versa. For eg - "2008,STEVEN,JOHNSON" is not present in RESULT but is present in DATA. And the entries should be made in RESULT dataset. The columns {SUBJECT, SCORE, SPORTS, DATASCIENCE} are made by my intuition that "spark" refers to the SUBJECT and so on. 
  Hope you understand my query. And I am using spark-shell with spark dataframes. 
  Note that "Spark" and "spark" should be considered as same.</p>