title: How to find exact median for grouped data in Spark
body: <p>I have a requirement to calculate exact median on grouped data set of Double datatype in Spark using Scala. </p>
  
  <p>It is different from the similar query: <a href="https://stackoverflow.com/questions/41404041">Find median in spark SQL for multiple double datatype columns</a>. This question is about the finding data for grouped data, whereas the other one is about finding median on a RDD level.</p>
  
  <p>Here is my sample data</p>
  
  <pre><code>scala&gt; sqlContext.sql("select * from test").show()
  
  +---+---+
  | id|num|
  +---+---+
  |  A|0.0|
  |  A|1.0|
  |  A|1.0|
  |  A|1.0|
  |  A|0.0|
  |  A|1.0|
  |  B|0.0|
  |  B|1.0|
  |  B|1.0|
  +---+---+
  </code></pre>
  
  <p>Expected Answer:</p>
  
  <pre><code>+--------+
  | Median |
  +--------+
  |   1    |
  |   1    |
  +--------+
  </code></pre>
  
  <p>I tried the following option, but no luck:</p>
  
  <p>1) Hive function percentile, it worked only for BigInt.</p>
  
  <p>2) Hive function <code>percentile_approx</code>, but it does not work as expected (returns 0.25 vs 1).</p>
  
  <pre><code>scala&gt; sqlContext.sql("select percentile_approx(num, 0.5) from test group by id").show()
  
  +----+
  | _c0|
  +----+
  |0.25|
  |0.25|
  +----+
  </code></pre>