title: How to lemmatise a dataframe column Python
body: <p>How can lemmatise a dataframe column. CSV file "train.csv" looks like this</p>
  
  <pre><code>id  tweet
  1   retweet if you agree
  2   happy birthday your majesty
  3   essential oils are not made of chemicals
  </code></pre>
  
  <p>I performed the following</p>
  
  <pre><code>import pandas as pd
  from nltk.tokenize import TweetTokenizer
  from nltk.corpus import stopwords
  from nltk.stem.wordnet import WordNetLemmatizer
  
  train_data = pd.read_csv('train.csv', error_bad_lines=False)
  print(train_data)
  
  # Removing stop words
  stop = stopwords.words('english')
  test = pd.DataFrame(train_data['tweet'])
  test.columns = ['tweet']
  
  test['tweet_without_stopwords'] = test['tweet'].apply(lambda x: ' '.join([word for word in x.split() if word not in (stop)]))
  print(test['tweet_without_stopwords'])
  
  # TOKENIZATION
  tt = TweetTokenizer()
  test['tokenised_tweet'] = test['tweet_without_stopwords'].apply(tt.tokenize)
  print(test)
  
  </code></pre>
  
  <p>output:</p>
  
  <pre><code>0 retweet if you agree ... [retweet, agree]
  1 happy birthday your majesty ... [happy, birthday, majesty]
  2 essential oils are not made of chemicals ... [essential, oils, made, chemicals]
  
  
  </code></pre>
  
  <p>I tried the following to lemmatise but I'm getting this error <em>TypeError: unhashable type: 'list'</em></p>
  
  <pre><code>
  lmtzr = WordNetLemmatizer()
  lemmatized = [[lmtzr.lemmatize(word) for word in test['tokenised_tweet']]]
  print(lemmatized)
  
  </code></pre>