title: Output of foreach function on Spark DataFrame
body: <p>I have seen various postings around (such as the answer to this <a href="https://stackoverflow.com/questions/25295277/view-rdd-contents-in-python-spark">stackexchange post</a>) that give something similar to the code below as a simple example of how to use the <code>foreach()</code> function on a Spark DataFrame.</p>
  
  <pre><code>spark = pyspark.sql.SparkSession.builder.getOrCreate()
  dftest = spark.createDataFrame([(1,"foo"), (2, "bar"), (3, "baz"), (4, "qux")]).toDF("number", "name")
  def g(x): print(x)
  dftest.groupBy("name").count().foreach(g)
  </code></pre>
  
  <p>Most of the postings I have seen show that the output should be a printout to the console of the count of each row in this simple example. But this does nothing for me.  It runs successfully without throwing any errors, but it does nothing.  </p>
  
  <p>I have tried on an Cloudera cluster, and on a local instance of Spark in a Jupyter notebook where the Master is <code>local[*]</code>. Both running Spark 2.2.0. Same results on both.</p>
  
  <p>What am I doing wrong?</p>
  
  <p><a href="https://spark.apache.org/docs/2.2.0/api/python/pyspark.sql.html#pyspark.sql.DataFrame.foreach" rel="nofollow noreferrer">Spark docs on <code>foreach()</code></a></p>
  
  <p>Using a for loop on the <code>collect()</code>'ed DataFrame, as one of the answers in this <a href="https://stackoverflow.com/questions/25295277/view-rdd-contents-in-python-spark">post</a> suggests, does not get to the problem I am trying to solve.  <code>collect()</code> returns a python list which you can then iterate over with python, it is not the same thing as iterating over, or applying a Spark function over every row DataFrame.</p>