title: How to count a list of rows words
body: <p>I am trying to do a word counting of the 100 first rows of my data frame df5.  I am trying to get this kind of result</p>
  <p><a href="https://i.stack.imgur.com/u7cHX.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/u7cHX.png" alt="enter image description here" /></a></p>
  <pre class="lang-py prettyprint-override"><code>#filterin first 100 rows
  df6 = df5.select(&quot;text_&quot;)
  df6_100rows = df6.head(100)
  #this save a list of rows
  print(df6_100rows)
  #[Row(text_='&quot;@MJCH85 Just say &quot;&quot;Alexa'), Row(text_='@ChandleeStyle That is defi
  
  #word count
  #I convert my list to RDD
  from pyspark import SparkContext
  sc =SparkContext.getOrCreate()
  
  #JAVA
  import os
  os.environ[&quot;JAVA_HOME&quot;] = &quot;/usr/lib/jvm/java-8-openjdk-amd64&quot;
  os.environ[&quot;SPARK_HOME&quot;] = &quot;/content/drive/MyDrive/IDS561_BigData/spark-3.1.2-bin-hadoop3.2&quot;
  
  #Counting the words in the data frame
  counts = rdd100rows.flatMap(lambda line: line.split(&quot; &quot;)).map(lambda word: (word, 1)).reduceByKey(lambda a, b: a + b).collect()
  print(counts)
  </code></pre>
  <p>I get an error which I presume has something to do with the fact that I got a list of rows (the code works well with an RDD made from a regular list)... but I am not sure about this or how to fix it. Also, I am working in collab.</p>
  <p><a href="https://i.stack.imgur.com/WQ2DI.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/WQ2DI.png" alt="enter image description here" /></a></p>