title: How to avoid using groupBy to find average of a column on a spark dataframe using dataframe APIs?
body: <p>I have a dataframe with the below data and columns:</p>
  <pre><code>sales_df.select('sales', 'monthly_sales').show()
  </code></pre>
  <blockquote>
  <pre><code>+----------------+--------------------------+
  |      sales     |        monthly_sales     |
  +----------------+--------------------------+
  |             mid|                      50.0|
  |             low|                      21.0|
  |             low|                      25.0|
  |            high|                      70.0|
  |             mid|                      60.0|
  |            high|                      75.0|
  |            high|                      95.0|
  |................|..........................|
  |................|..........................|
  |................|..........................|    
  |             low|                      25.0|
  |             low|                      20.0|
  +----------------+--------------------------+
  </code></pre>
  </blockquote>
  <p>I am trying to find the average of each sales type into a dataframe where I only have three rows(one for each sales type) in my final dataframe.
  <code>sale &amp; average_sale</code></p>
  <p>I used <code>groupBy</code> to achieve this.</p>
  <pre><code>sales_df.groupBy(&quot;sales&quot;).avg(&quot;monthly_sales&quot;).alias('average_sales').show()
  </code></pre>
  <p>and I was able to get the average sale as well.</p>
  <blockquote>
  <pre><code>+----------------+-------------------------------+
  |      sales     |          average sales        |
  +----------------+-------------------------------+
  |             mid|              5.568177828054298|
  |            high|              1.361184210526316|
  |             low|              3.014350758853288|
  +----------------+-------------------------------+
  </code></pre>
  </blockquote>
  <p>This ran faster because I am running my logic on test data which has 200 rows. So the code ran in no time. But I have huge data in my actual application and then there is the problem of  data shuffle due to groupBy.</p>
  <p>Is there any better way to find out the average without using <code>groupBy</code> ?
  Could anyone let me know the efficient way to achieve the solution considering huge data size.</p>