title: SkLearn Multinomial NB: Most Informative Features
body: <p>As my classifier yields about 99% accuracy on test data, I am a bit suspicious and want to gain insight in the most informative features of my NB classifier to see what kind of features it is learning. The following topic has been very useful: <a href="https://stackoverflow.com/questions/11116697/how-to-get-most-informative-features-for-scikit-learn-classifiers">How to get most informative features for scikit-learn classifiers?</a></p>
  
  <p>As for my feature input, I am still playing around and at the moment I am testing a simple unigram model, using CountVectorizer:</p>
  
  <pre><code> vectorizer = CountVectorizer(ngram_range=(1, 1), min_df=2, stop_words='english')
  </code></pre>
  
  <p>On the aforementioned topic I found the following function:</p>
  
  <pre><code>def show_most_informative_features(vectorizer, clf, n=20):
  feature_names = vectorizer.get_feature_names()
  coefs_with_fns = sorted(zip(clf.coef_[0], feature_names))
  top = zip(coefs_with_fns[:n], coefs_with_fns[:-(n + 1):-1])
  for (coef_1, fn_1), (coef_2, fn_2) in top:
      print "\t%.4f\t%-15s\t\t%.4f\t%-15s" % (coef_1, fn_1, coef_2, fn_2)
  </code></pre>
  
  <p>Which gives the following result:</p>
  
  <pre><code>    -16.2420        114th                   -4.0020 said           
      -16.2420        115                     -4.6937 obama          
      -16.2420        136                     -4.8614 house          
      -16.2420        14th                    -5.0194 president      
      -16.2420        15th                    -5.1236 state          
      -16.2420        1600                    -5.1370 senate         
      -16.2420        16th                    -5.3868 new            
      -16.2420        1920                    -5.4004 republicans    
      -16.2420        1961                    -5.4262 republican     
      -16.2420        1981                    -5.5637 democrats      
      -16.2420        19th                    -5.6182 congress       
      -16.2420        1st                     -5.7314 committee      
      -16.2420        31st                    -5.7732 white          
      -16.2420        3rd                     -5.8227 security       
      -16.2420        4th                     -5.8256 states         
      -16.2420        5s                      -5.8530 year           
      -16.2420        61                      -5.9099 government     
      -16.2420        900                     -5.9464 time           
      -16.2420        911                     -5.9984 department     
      -16.2420        97                      -6.0273 gop 
  </code></pre>
  
  <p>It works, but I would like to know what this function does in order to interpret the results. Mostly, I struggle with what the 'coef_' attribute does.</p>
  
  <p>I understand that the left side is the top 20 feature names with lowest coefficients, and the right side the features with the highest coefficients. But how exactly does this work, how do I interpret this overview? Does it mean that the left side holds the most informative features for the negative class, and the right side the most informative features for the positive class?</p>
  
  <p>Also, on the left side it kind of looks as if the feature names are sorted alphabetically, is this correct? </p>