title: TypeError: sparse matrix length is ambiguous; use getnnz() or shape[0] while using RF classifier?
body: <p>I am learning about random forests in scikit learn and as an example I would like to use Random forest classifier for text classification, with my own dataset. So first I vectorized the text with tfidf and for classification:</p>
  
  <pre><code>from sklearn.ensemble import RandomForestClassifier
  classifier=RandomForestClassifier(n_estimators=10) 
  classifier.fit(X_train, y_train)           
  prediction = classifier.predict(X_test)
  </code></pre>
  
  <p>When I run the classification I got this:</p>
  
  <pre><code>TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.
  </code></pre>
  
  <p>then I used the <code>.toarray()</code> for <code>X_train</code> and I got the following:</p>
  
  <pre><code>TypeError: sparse matrix length is ambiguous; use getnnz() or shape[0]
  </code></pre>
  
  <p>From a previous <a href="https://stackoverflow.com/questions/21689141/classifying-text-documents-with-random-forests">question</a> as I understood I need to reduce the dimensionality of the numpy array so I do the same:</p>
  
  <pre><code>from sklearn.decomposition.truncated_svd import TruncatedSVD        
  pca = TruncatedSVD(n_components=300)                                
  X_reduced_train = pca.fit_transform(X_train)               
  
  from sklearn.ensemble import RandomForestClassifier                 
  classifier=RandomForestClassifier(n_estimators=10)                  
  classifier.fit(X_reduced_train, y_train)                            
  prediction = classifier.predict(X_testing) 
  </code></pre>
  
  <p>Then I got this exception:</p>
  
  <pre><code>  File "/usr/local/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 419, in predict
      n_samples = len(X)
    File "/usr/local/lib/python2.7/site-packages/scipy/sparse/base.py", line 192, in __len__
      raise TypeError("sparse matrix length is ambiguous; use getnnz()"
  TypeError: sparse matrix length is ambiguous; use getnnz() or shape[0]
  </code></pre>
  
  <p>The I tried the following:</p>
  
  <pre><code>prediction = classifier.predict(X_train.getnnz()) 
  </code></pre>
  
  <p>And got this:</p>
  
  <pre><code>  File "/usr/local/lib/python2.7/site-packages/sklearn/ensemble/forest.py", line 419, in predict
      n_samples = len(X)
  TypeError: object of type 'int' has no len()
  </code></pre>
  
  <p>Two questions were raised from this: How can I use Random forests to classify correctly? and what's happening with <code>X_train</code>?. </p>
  
  <p>Then I tried the following:</p>
  
  <pre><code>df = pd.read_csv('/path/file.csv',
  header=0, sep=',', names=['id', 'text', 'label'])
  
  
  
  X = tfidf_vect.fit_transform(df['text'].values)
  y = df['label'].values
  
  
  
  from sklearn.decomposition.truncated_svd import TruncatedSVD
  pca = TruncatedSVD(n_components=2)
  X = pca.fit_transform(X)
  
  a_train, a_test, b_train, b_test = train_test_split(X, y, test_size=0.33, random_state=42)
  
  from sklearn.ensemble import RandomForestClassifier
  
  classifier=RandomForestClassifier(n_estimators=10)
  classifier.fit(a_train, b_train)
  prediction = classifier.predict(a_test)
  
  from sklearn.metrics.metrics import precision_score, recall_score, confusion_matrix, classification_report
  print '\nscore:', classifier.score(a_train, b_test)
  print '\nprecision:', precision_score(b_test, prediction)
  print '\nrecall:', recall_score(b_test, prediction)
  print '\n confussion matrix:\n',confusion_matrix(b_test, prediction)
  print '\n clasification report:\n', classification_report(b_test, prediction)
  </code></pre>