title: How term frequency is calculated in TfidfVectorizer?
body: <p>I searched a lot for understanding this but I am not able to. I understand that by default TfidfVectorizer will apply <code>l2</code> normalization on term frequency. <a href="http://blog.christianperone.com/2011/10/machine-learning-text-feature-extraction-tf-idf-part-ii/" rel="nofollow noreferrer">This</a> article explain the equation of it. I am using TfidfVectorizer on my text written in Gujarati language. Following is details of output about it:</p>
  
  <p>My two documents are:</p>
  
  <pre><code>ખુબ વખાણ કરે છે
  
  ખુબ વધારે છે
  </code></pre>
  
  <p>The code I am using is:</p>
  
  <pre><code>vectorizer = TfidfVectorizer(tokenizer=tokenize_words, sublinear_tf=True, use_idf=True, smooth_idf=False)
  </code></pre>
  
  <p>Here, <code>tokenize_words</code> is my function for tokenizing words.
  The list of TF-IDF of my data is:</p>
  
  <pre><code>[[ 0.6088451   0.35959372  0.35959372  0.6088451   0.        ]
   [ 0.          0.45329466  0.45329466  0.          0.76749457]]
  </code></pre>
  
  <p>The list of features:</p>
  
  <pre><code>['કરે', 'ખુબ', 'છે.', 'વખાણ', 'વધારે']
  </code></pre>
  
  <p>The value of idf:</p>
  
  <pre><code>{'વખાણ': 1.6931471805599454, 'છે.': 1.0, 'કરે': 1.6931471805599454, 'વધારે': 1.6931471805599454, 'ખુબ': 1.0}
  </code></pre>
  
  <p>Please explain me in this example what shall be the term frequency of each term in my both documents.</p>