title: Keep TFIDF result for predicting new content using Scikit for Python
body: <p>I am using sklearn on Python to do some clustering. I've trained 200,000 data, and code below works well.</p>
  
  <pre><code>corpus = open("token_from_xml.txt")
  vectorizer = CountVectorizer(decode_error="replace")
  transformer = TfidfTransformer()
  tfidf = transformer.fit_transform(vectorizer.fit_transform(corpus))
  km = KMeans(30)
  kmresult = km.fit(tfidf).predict(tfidf)
  </code></pre>
  
  <p>But when I have new testing content, I'd like to cluster it to existed clusters I'd trained. So I'm wondering how to <strong>save IDF result, so that I can do TFIDF for the new testing content and make sure the result for new testing content have same array length.</strong></p>
  
  <p>Thanks in advance.</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>I may need to save "transformer" or "tfidf" variable to file(txt or others), if one of them contains the trained IDF result.</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>For example. I have the training data:</p>
  
  <pre><code>["a", "b", "c"]
  ["a", "b", "d"]
  </code></pre>
  
  <p>And do TFIDF, the result will contains 4 features(a,b,c,d)</p>
  
  <p>When I <strong>TEST</strong>:</p>
  
  <pre><code>["a", "c", "d"]
  </code></pre>
  
  <p>to see which cluster(already made by k-means) it belongs to. TFIDF will only give the result with 3 features(a,c,d), so the clustering in k-means will fall. (If I test <code>["a", "b", "e"]</code>, there may have other problems.)</p>
  
  <p>So how to store the features list for testing data (even more, store it in file)?</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>Solved, see answers below.</p>