title: text vectorizer using sckitlearn
body: <p>i am using sckit-learn to do preprocessing on text data my aim is to get vector representation of my data (features and label) , what i have done is vectorizing the feature using tfidf after that the dataset size changed by double size as <strong>X.ravel()</strong> is used .</p>
  
  <p>X before (30376, 2) </p>
  
  <p>X after (60752, 41331) </p>
  
  <p>my problem is that when i am having two features in the x vector and i want to get vector representation correctly how i can do it </p>
  
  <pre><code> df = pd.read_csv('Dataset.csv',encoding='latin1')
      df = df.dropna()
  
  
  
      X = np.array(df.drop(['Type'], 1))
      y = np.array(df['Type'])
  
  
  
  
      #print(X)
  
      print("Extracting features from the training data using a sparse vectorizer")
      vectorizer= TfidfVectorizer(sublinear_tf=True, max_df=0.5,
                                       stop_words='english')
      X = vectorizer.fit_transform(X.ravel().astype('U'))
  
      imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
      X = imp.fit_transform(X)
      X.shape
      X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
  </code></pre>
  
  <p>and if that the case when i am using <strong>train_test_split</strong> i get this error i don't understand what it referring to </p>
  
  <pre><code>TypeError: Singleton array array(TfidfVectorizer(analyzer='word', binary=False, decode_error='strict',
          dtype=&lt;class 'numpy.int64'&gt;, encoding='utf-8', input='content',
          lowercase=True, max_df=0.5, max_features=None, min_df=1,
          ngram_range=(1, 1), norm='l2', preprocessor=None, smooth_idf=True,
          stop_words='english', strip_accents=None, sublinear_tf=True,
          token_pattern='(?u)\\b\\w\\w+\\b', tokenizer=None, use_idf=True,
          vocabulary=None), dtype=object) cannot be considered a valid collection.
  </code></pre>
  
  <p>any suggestion and thanks </p>