title: Iterate nltk.tokenize across all rows of a pandas dataframe
body: <p>grateful for your help for what feels like a stupid question.  I've pulled a sqlite table into a pandas dataframe so I can tokenize and count the frequency of words from a series of tweets. </p>
  
  <p>With the code below, I can produce this for the first tweet.  How do I iterate for the whole table?  </p>
  
  <pre><code>conn = sqlite3.connect("tweets.sqlite")
  data = pd.read_sql_query("select tweet_text from tweets_new;", conn)
  
  tokenizer=RegexpTokenizer(r'\w+')
  tokens=tokenizer.tokenize(data['tweet_text'][0])
  
  words = nltk.FreqDist(tokens)
  
  unigram_df = pd.DataFrame(words.most_common(),
                               columns=["WORD","COUNT"])
  
  unigram_df
  </code></pre>
  
  <p>When I change the value to anything other than a single row, I get the following error:</p>
  
  <pre><code>TypeError: expected string or buffer
  </code></pre>
  
  <p>I know there are other ways of doing this, but I need to do it along these lines because of how I intend to use the output next.  Thanks for any help you can provide!</p>
  
  <p>I have tried:</p>
  
  <pre><code>%%time
  
  tokenizer = RegexpTokenizer(r'\w+')  
  
  print "Cleaning the tweets...\n"
  for i in xrange(0,len(df)):
      if( (i+1)%1000000 == 0 ):  
          tokens=tokenizer.tokenize(df['tweet_text'][i])
          words = nltk.FreqDist(tokens)
  </code></pre>
  
  <p>This looks like it should work, but still only returns words from the first row.</p>