title: How to save sklearn pipeline/feature-transformer
body: <p>I have a pipeline contains only a feature union that has three different sets of feature, including tfidf:</p>
  
  <pre><code>A_vec = AVectorizer()
  B_vec = BVectorizer()
  tfidf_vec = TfidfVectorizer(ngram_range=(1,2), analyzer='word', binary=False, stop_words=stopWords, min_df=0.01, use_idf=True)
  all_features = FeatureUnion([('A_feature', A_vec), ('V_feature', B_vec), ('tfidf_feature', tfidf_vec)])
  pipeline = Pipeline([('all_feature', all_features)])
  </code></pre>
  
  <p>I want to save this pipelined feature transformer for my test data (I am using LibSVM for classification), and this is what I have tried:</p>
  
  <ul>
  <li><p>I have used joblib.dump to save this pipeline but it generated toooo many .npy files so I had to stop the writing process. It was a rather stupid attempt!</p></li>
  <li><p>I have saved tfidf_vec.vocabulary_ and thus</p>
  
  <p>tfidf_vec2 = TfidfVectorizer(ngram_range=(1,3), analyzer='word', binary=False, stop_words=stopWords, min_df=0.01, use_idf=True,vocabulary=pickle.load(open("../vocab.pkl", "rb"))</p>
  
  <p>... ...</p>
  
  <p>feat_test = pipeline2.transform(X_test)</p></li>
  </ul>
  
  <p>It says "NotFittedError: idf vector is not fitted". I then used fit_transform rather than transform but it generates a feature vector that contains different values (comparing to the correct feature vector). Then I followed <a href="http://thiagomarzagao.com/2015/12/08/saving-TfidfVectorizer-without-pickles/" rel="nofollow">http://thiagomarzagao.com/2015/12/08/saving-TfidfVectorizer-without-pickles/</a> and still struggling to get it work. </p>
  
  <p>Is there a simpler way to achieve this? Thanks!</p>