title: aggregate function Count usage with groupBy in Spark
body: <p>I'm trying to make multiple operations in one line of code in pySpark,
  and not sure if that's possible for my case.</p>
  
  <p>My intention is not having to save the output as a new dataframe.</p>
  
  <p>My current code is rather simple:</p>
  
  <pre><code>encodeUDF = udf(encode_time, StringType())
  new_log_df.cache().withColumn('timePeriod', encodeUDF(col('START_TIME')))
    .groupBy('timePeriod')
    .agg(
      mean('DOWNSTREAM_SIZE').alias("Mean"),
      stddev('DOWNSTREAM_SIZE').alias("Stddev")
    )
    .show(20, False)
  </code></pre>
  
  <p>And my intention is to add <code>count()</code> after using <code>groupBy</code>, to get, well, the count of records matching each value of <strong><code>timePeriod</code></strong> column, printed\shown as output.</p>
  
  <p>When trying to use <code>groupBy(..).count().agg(..)</code> I get exceptions.</p>
  
  <p><strong>Is there any way to achieve both <code>count()</code> and <code>agg()</code></strong>.show() prints, without splitting code to two lines of commands, e.g. :</p>
  
  <pre><code>new_log_df.withColumn(..).groupBy(..).count()
  new_log_df.withColumn(..).groupBy(..).agg(..).show()
  </code></pre>
  
  <p>Or better yet, for getting a merged output to <code>agg.show()</code> output - An extra column which states the counted number of records matching the row's value. e.g.:</p>
  
  <pre><code>timePeriod | Mean | Stddev | Num Of Records
      X      | 10   |   20   |    315
  </code></pre>