title: Custom Transformer and FeatureUnion for word2vec
body: <p>I am trying to classify a set of text documents using multiple sets of features. I am using <a href="http://scikit-learn.org/stable/auto_examples/hetero_feature_union.html#sphx-glr-auto-examples-hetero-feature-union-py" rel="nofollow noreferrer">sklearn's Feature Union</a> to combine different features for fitting into a single model. One of the features includes word embeddings using <a href="https://radimrehurek.com/gensim/models/word2vec.html" rel="nofollow noreferrer">gensim's word2vec</a>.</p>
  
  <pre><code>import numpy as np
  from gensim.models.word2vec import Word2Vec
  from sklearn.pipeline import FeatureUnion
  from sklearn.pipeline import Pipeline
  from sklearn.linear_model import SGDClassifier
  from sklearn.decomposition import TruncatedSVD
  from sklearn.feature_extraction.text import CountVectorizer
  from sklearn.feature_extraction.text import TfidfTransformer
  from sklearn.base import BaseEstimator, TransformerMixin
  from sklearn.datasets import fetch_20newsgroups
  from sklearn.feature_selection import chi2
  from sklearn.feature_selection import SelectKBest
  
  categories = ['alt.atheism', 'talk.religion.misc', 'comp.graphics', 'sci.space']
  data = fetch_20newsgroups(subset='train', categories=categories)#dummy dataset
  
  w2v_model= Word2Vec(data .data, size=100, window=5, min_count=5, workers=2)
  word2vec={w: vec for w, vec in zip(w2v_model.wv.index2word, w2v_model.wv.syn0)} #dictionary of word embeddings
  feat_select = SelectKBest(score_func=chi2, k=10) #other features
  TSVD = TruncatedSVD(n_components=50, algorithm = "randomized", n_iter = 5)
  #other features
  </code></pre>
  
  <p>In order to include transformers/estimators not already available in sklearn, I am attempting to wrap my word2vec results into a custom transformer class that returns the vector averages. </p>
  
  <pre><code>class w2vTransformer(TransformerMixin):
      """
      Wrapper class for running word2vec into pipelines and FeatureUnions
      """
      def __init__(self,word2vec,**kwargs):
          self.word2vec=word2vec
          self.kwargs=kwargs
          self.dim = len(word2vec.values())
      def fit(self,x, y=None):
          return self
  
      def transform(self, X):
          return np.array([
          np.mean([self.word2vec[w] for w in words if w in self.word2vec] 
              or [np.zeros(self.dim)], axis=0)
         for words in X
  ])
  </code></pre>
  
  <p>However when it comes time to fit the model I receive an error.</p>
  
  <pre><code>combined_features = FeatureUnion([("w2v_class",w2vTransformer(word2vec)),
       ("feat",feat_select),("TSVD",TSVD)])#join features into combined_features
  #combined_features = FeatureUnion([("feat",feat_select),("TSVD",TSVD)])#runs when word embeddings are not included    
  text_clf_svm = Pipeline([('vect', CountVectorizer()),
           ('tfidf', TfidfTransformer()),
           ('feature_selection', combined_features),
            ('clf-svm',  SGDClassifier( loss="modified_huber")),
   ]) 
  
  text_clf_svm_1 = text_clf_svm.fit(data.data,data.target) # fits data
  
  text_clf_svm_1 = text_clf_svm.fit(data.data,data.target) # fits data
  Traceback (most recent call last):
  
    File "&lt;ipython-input-8-a085b7d40f8f&gt;", line 1, in &lt;module&gt;
      text_clf_svm_1 = text_clf_svm.fit(data.data,data.target) # fits data
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 248, in fit
      Xt, fit_params = self._fit(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 213, in _fit
      **fit_params_steps[name])
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\memory.py", line 362, in __call__
      return self.func(*args, **kwargs)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 581, in _fit_transform_one
      res = transformer.fit_transform(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 739, in fit_transform
      for name, trans, weight in self._iter())
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 779, in __call__
      while self.dispatch_one_batch(iterator):
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 625, in dispatch_one_batch
      self._dispatch(tasks)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 588, in _dispatch
      job = self._backend.apply_async(batch, callback=cb)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 111, in apply_async
      result = ImmediateResult(func)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 332, in __init__
      self.results = batch()
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in __call__
      return [func(*args, **kwargs) for func, args, kwargs in self.items]
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in &lt;listcomp&gt;
      return [func(*args, **kwargs) for func, args, kwargs in self.items]
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 581, in _fit_transform_one
      res = transformer.fit_transform(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\base.py", line 520, in fit_transform
      return self.fit(X, y, **fit_params).transform(X)
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 16, in transform
      for words in X
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 16, in &lt;listcomp&gt;
      for words in X
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 14, in &lt;listcomp&gt;
      np.mean([self.word2vec[w] for w in words if w in self.word2vec]
  
  TypeError: unhashable type: 'csr_matrix'
  
  Traceback (most recent call last):
  
    File "&lt;ipython-input-8-a085b7d40f8f&gt;", line 1, in &lt;module&gt;
      text_clf_svm_1 = text_clf_svm.fit(data.data,data.target) # fits data
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 248, in fit
      Xt, fit_params = self._fit(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 213, in _fit
      **fit_params_steps[name])
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\memory.py", line 362, in __call__
      return self.func(*args, **kwargs)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 581, in _fit_transform_one
      res = transformer.fit_transform(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 739, in fit_transform
      for name, trans, weight in self._iter())
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 779, in __call__
      while self.dispatch_one_batch(iterator):
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 625, in dispatch_one_batch
      self._dispatch(tasks)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 588, in _dispatch
      job = self._backend.apply_async(batch, callback=cb)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 111, in apply_async
      result = ImmediateResult(func)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 332, in __init__
      self.results = batch()
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in __call__
      return [func(*args, **kwargs) for func, args, kwargs in self.items]
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in &lt;listcomp&gt;
      return [func(*args, **kwargs) for func, args, kwargs in self.items]
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\pipeline.py", line 581, in _fit_transform_one
      res = transformer.fit_transform(X, y, **fit_params)
  
    File "C:\Users\rlusk\AppData\Local\Continuum\Anaconda3\lib\site-packages\sklearn\base.py", line 520, in fit_transform
      return self.fit(X, y, **fit_params).transform(X)
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 16, in transform
      for words in X
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 16, in &lt;listcomp&gt;
      for words in X
  
    File "&lt;ipython-input-6-cbc52cd420cd&gt;", line 14, in &lt;listcomp&gt;
      np.mean([self.word2vec[w] for w in words if w in self.word2vec]
  
  TypeError: unhashable type: 'csr_matrix'
  </code></pre>
  
  <p>I understand that the error is because the variable "words" is a csr_matrix, but it needs to be an iterable such as a list. My question is how do I modify the transformer class or data so I can use the word embeddings as features to feed into FeatureUnion? This is my first SO post, please be gentle. </p>