<p>This is a great discussion. Thanks for starting this thread. The idea of including document length by @avip seems interesting. Will have to experiment and check on the results. In the meantime let me try asking the question a little differently. What are we trying to interpret when querying for TF-IDF relevance scores ?</p>

<ol>
<li>Possibly trying to understand the word relevance at the document level </li>
<li>Possibly trying to understand the word relevance per Class</li>
<li><p>Possibly trying to understand the word relevance overall ( in the whole 
corpus )</p>

<pre><code> # # features, corpus = 6 documents of length 3
 counts = [[3, 0, 1],
           [2, 0, 0],
           [3, 0, 0],
           [4, 0, 0],
           [3, 2, 0],
           [3, 0, 2]]
 from sklearn.feature_extraction.text import TfidfTransformer
 transformer = TfidfTransformer(smooth_idf=False)
 tfidf = transformer.fit_transform(counts)
 print(tfidf.toarray())

 # lambda for basic stat computation
 summarizer_default = lambda x: np.sum(x, axis=0)
 summarizer_mean = lambda x: np.mean(x, axis=0)

 print(summarizer_default(tfidf))
 print(summarizer_mean(tfidf))
</code></pre></li>
</ol>

<p>Result:</p>

<pre><code># Result post computing TF-IDF relevance scores
array([[ 0.81940995,  0.        ,  0.57320793],
           [ 1.        ,  0.        ,  0.        ],
           [ 1.        ,  0.        ,  0.        ],
           [ 1.        ,  0.        ,  0.        ],
           [ 0.47330339,  0.88089948,  0.        ],
           [ 0.58149261,  0.        ,  0.81355169]])

# Result post aggregation (Sum, Mean) 
[[ 4.87420595  0.88089948  1.38675962]]
[[ 0.81236766  0.14681658  0.2311266 ]]
</code></pre>

<p>If we observe closely, we realize the the feature1 witch occurred in all the document is not ignored completely because the sklearn implementation of idf = log [ n / df(d, t) ] + 1. +1 is added so that the important word which just so happens to occur in all document is not ignored. E.g. the word 'bike' occurring very frequently in classifying a particular document as 'motorcyle' (20_newsgroup dataset).</p>

<ol>
<li><p>Now in-regards to the first 2 questions, one is trying to interpret and understand the top common features that might be occurring in the document. In that case, aggregating in some form including all possible occurrence of the word in a doc is not taking anything away even mathematically. IMO such a query is very useful exploring the dataset and helping understanding what the dataset is about. The logic might be applied to vectorizing using Hashing as well.</p>

<p>relevance_score = mean(tf(t,d) * idf(t,d)) = mean( (bias + 
            inital_wt * F(t,d) / max{F(t',d)}) * log(N/df(d, t)) + 1 ))</p></li>
<li><p>Question3 is very important as it might as well be contributing to 
features being selected for building a predictive model. Just using TF-IDF scores independently for feature selection might be misleading at multiple level. Adopting a more theoretical statistical test such as 'chi2' couple with TF-IDF relevance scores might be a better approach. Such statistical test also evaluates the importance of the feature in relation to the respective target class.</p></li>
</ol>

<p>And of-course combining such interpretation with the model's learned feature weights would be very helpful in understanding the importance of text derived features completely.</p>

<p>** The problem is a little more complex to cover in detail here. But, hoping the above helps. What do others feel ? </p>

<p>Reference: <a href="https://arxiv.org/abs/1707.05261" rel="nofollow noreferrer">https://arxiv.org/abs/1707.05261</a></p>
Interpreting the sum of TF-IDF scores of words across documents
<p>First let's extract the TF-IDF scores per term per document:</p>

<pre><code>from gensim import corpora, models, similarities
documents = ["Human machine interface for lab abc computer applications",
              "A survey of user opinion of computer system response time",
              "The EPS user interface management system",
              "System and human system engineering testing of EPS",
              "Relation of user perceived response time to error measurement",
              "The generation of random binary unordered trees",
              "The intersection graph of paths in trees",
              "Graph minors IV Widths of trees and well quasi ordering",
              "Graph minors A survey"]
stoplist = set('for a of the and to in'.split())
texts = [[word for word in document.lower().split() if word not in stoplist] for document in documents]
dictionary = corpora.Dictionary(texts)
corpus = [dictionary.doc2bow(text) for text in texts]
tfidf = models.TfidfModel(corpus)
corpus_tfidf = tfidf[corpus]
</code></pre>

<p>Printing it out:</p>

<pre><code>for doc in corpus_tfidf:
    print doc
</code></pre>

<p>[out]:</p>

<pre><code>[(0, 0.4301019571350565), (1, 0.4301019571350565), (2, 0.4301019571350565), (3, 0.4301019571350565), (4, 0.2944198962221451), (5, 0.2944198962221451), (6, 0.2944198962221451)]
[(4, 0.3726494271826947), (7, 0.27219160459794917), (8, 0.3726494271826947), (9, 0.27219160459794917), (10, 0.3726494271826947), (11, 0.5443832091958983), (12, 0.3726494271826947)]
[(6, 0.438482464916089), (7, 0.32027755044706185), (9, 0.32027755044706185), (13, 0.6405551008941237), (14, 0.438482464916089)]
[(5, 0.3449874408519962), (7, 0.5039733231394895), (14, 0.3449874408519962), (15, 0.5039733231394895), (16, 0.5039733231394895)]
[(9, 0.21953536176370683), (10, 0.30055933182961736), (12, 0.30055933182961736), (17, 0.43907072352741366), (18, 0.43907072352741366), (19, 0.43907072352741366), (20, 0.43907072352741366)]
[(21, 0.48507125007266594), (22, 0.48507125007266594), (23, 0.48507125007266594), (24, 0.48507125007266594), (25, 0.24253562503633297)]
[(25, 0.31622776601683794), (26, 0.31622776601683794), (27, 0.6324555320336759), (28, 0.6324555320336759)]
[(25, 0.20466057569885868), (26, 0.20466057569885868), (29, 0.2801947048062438), (30, 0.40932115139771735), (31, 0.40932115139771735), (32, 0.40932115139771735), (33, 0.40932115139771735), (34, 0.40932115139771735)]
[(8, 0.6282580468670046), (26, 0.45889394536615247), (29, 0.6282580468670046)]
</code></pre>

<p>If we want to find the "saliency" or "importance" of the words within this corpus, <strong>can we simple do the sum of the tf-idf scores across all documents and divide it by the number of documents?</strong> I.e. </p>

<pre><code>&gt;&gt;&gt; tfidf_saliency = Counter()
&gt;&gt;&gt; for doc in corpus_tfidf:
...     for word, score in doc:
...         tfidf_saliency[word] += score / len(corpus_tfidf)
... 
&gt;&gt;&gt; tfidf_saliency
Counter({7: 0.12182694202050007, 8: 0.11121194156107769, 26: 0.10886469856464989, 29: 0.10093919463036093, 9: 0.09022272408985754, 14: 0.08705221175200946, 25: 0.08482488519466996, 6: 0.08143359568202602, 10: 0.07480097322359022, 12: 0.07480097322359022, 4: 0.07411881371164887, 13: 0.07117278898823597, 5: 0.07104525967490458, 27: 0.07027283689263066, 28: 0.07027283689263066, 11: 0.060487023243988705, 15: 0.055997035904387725, 16: 0.055997035904387725, 21: 0.05389680556362955, 22: 0.05389680556362955, 23: 0.05389680556362955, 24: 0.05389680556362955, 17: 0.048785635947490406, 18: 0.048785635947490406, 19: 0.048785635947490406, 20: 0.048785635947490406, 0: 0.04778910634833961, 1: 0.04778910634833961, 2: 0.04778910634833961, 3: 0.04778910634833961, 30: 0.045480127933079706, 31: 0.045480127933079706, 32: 0.045480127933079706, 33: 0.045480127933079706, 34: 0.045480127933079706})
</code></pre>

<p>Looking at the output, could we assume that the most "prominent" word in the corpus is:</p>

<pre><code>&gt;&gt;&gt; dictionary[7]
u'system'
&gt;&gt;&gt; dictionary[8]
u'survey'
&gt;&gt;&gt; dictionary[26]
u'graph'
</code></pre>

<p>If so, <strong>what is the mathematical interpretation of the sum of TF-IDF scores of words across documents?</strong></p>