title: TfidfVectorizer in scikit-learn : ValueError: np.nan is an invalid document
body: <p>I'm using TfidfVectorizer from scikit-learn to do some feature extraction from text data. I have a CSV file with a Score (can be +1 or -1) and a Review (text). I pulled this data into a DataFrame so I can run the Vectorizer.</p>
  
  <p>This is my code: </p>
  
  <pre><code>import pandas as pd
  import numpy as np
  from sklearn.feature_extraction.text import TfidfVectorizer
  
  df = pd.read_csv("train_new.csv",
               names = ['Score', 'Review'], sep=',')
  
  # x = df['Review'] == np.nan
  #
  # print x.to_csv(path='FindNaN.csv', sep=',', na_rep = 'string', index=True)
  #
  # print df.isnull().values.any()
  
  v = TfidfVectorizer(decode_error='replace', encoding='utf-8')
  x = v.fit_transform(df['Review'])
  </code></pre>
  
  <p>This is the traceback for the error I get: </p>
  
  <pre><code>Traceback (most recent call last):
    File "/home/PycharmProjects/Review/src/feature_extraction.py", line 16, in &lt;module&gt;
  x = v.fit_transform(df['Review'])
   File "/home/b/hw1/local/lib/python2.7/site-   packages/sklearn/feature_extraction/text.py", line 1305, in fit_transform
     X = super(TfidfVectorizer, self).fit_transform(raw_documents)
   File "/home/b/work/local/lib/python2.7/site-packages/sklearn/feature_extraction/text.py", line 817, in fit_transform
  self.fixed_vocabulary_)
   File "/home/b/work/local/lib/python2.7/site- packages/sklearn/feature_extraction/text.py", line 752, in _count_vocab
     for feature in analyze(doc):
   File "/home/b/work/local/lib/python2.7/site-packages/sklearn/feature_extraction/text.py", line 238, in &lt;lambda&gt;
  tokenize(preprocess(self.decode(doc))), stop_words)
   File "/home/b/work/local/lib/python2.7/site-packages/sklearn/feature_extraction/text.py", line 118, in decode
   raise ValueError("np.nan is an invalid document, expected byte or "
   ValueError: np.nan is an invalid document, expected byte or unicode string.
  </code></pre>
  
  <p>I checked the CSV file and DataFrame for anything that's being read as NaN but I can't find anything. There are 18000 rows, none of which return <code>isnan</code> as True. </p>
  
  <p>This is what <code>df['Review'].head()</code> looks like: </p>
  
  <pre><code>  0    This book is such a life saver.  It has been s...
    1    I bought this a few times for my older son and...
    2    This is great for basics, but I wish the space...
    3    This book is perfect!  I'm a first time new mo...
    4    During your postpartum stay at the hospital th...
    Name: Review, dtype: object
  </code></pre>