title: Spark: Dynamic filter on a dataset in scala
body: <p>I have a dataset (<code>ds</code>) which looks like</p>
  
  <pre><code>scala&gt; ds.show()
  +----+---+-----+----+-----+--------------+
  |name|age|field|optr|value|          rule|
  +----+---+-----+----+-----+--------------+
  |   a| 75|  age|   &lt;|   18|         Minor|
  |   b| 10|  age|   &lt;|   18|         Minor|
  |   c| 30|  age|   &lt;|   18|         Minor|
  |   a| 75|  age|  &gt;=|   18|         Major|
  |   b| 10|  age|  &gt;=|   18|         Major|
  |   c| 30|  age|  &gt;=|   18|         Major|
  |   a| 75|  age|   &gt;|   60|Senior Citizen|
  |   b| 10|  age|   &gt;|   60|Senior Citizen|
  |   c| 30|  age|   &gt;|   60|Senior Citizen|
  +----+---+-----+----+-----+--------------+
  </code></pre>
  
  <p>Now I need to apply filter on this to get those rows which satisfies filter condition as specified below.</p>
  
  <ul>
  <li>Apply filter on the field in <code>field</code> column</li>
  <li>Operation to perform is in <code>optr</code> column, and</li>
  <li>Value to compare is in <code>value</code> column.<br></li>
  </ul>
  
  <p><strong>Example:</strong> For first row - Apply filter on <code>age</code> column (here all field values are age, but it can be different), where age is <em>less than (&lt;) value 18</em>, which is <code>false</code> as <em>age=75</em>.<br><br>
  I don't know how to specify this filter condition in scala. The resulting dataset should look like</p>
  
  <pre><code>+----+---+-----+----+-----+--------------+
  |name|age|field|optr|value|          rule|
  +----+---+-----+----+-----+--------------+
  |   b| 10|  age|   &lt;|   18|         Minor|
  |   a| 75|  age|  &gt;=|   18|         Major|
  |   c| 30|  age|  &gt;=|   18|         Major|
  |   a| 75|  age|   &gt;|   60|Senior Citizen|
  +----+---+-----+----+-----+--------------+
  </code></pre>