<p>The only <code>word2vec.c</code> command-line parameters which differentially affect words by their frequency are <code>min_count</code>, which discards words below a certain threshold, and <code>sample</code>, which randomly discards <em>some</em> occurrences of highly-frequent words.</p>
<p>You can do that discarding because using all occurrences of highly-frequent words is overkill: it barely improves their vectors over fewer training samples, it takes extra training time, &amp; it essentially dilutes the influence of rarer words on the model's internal shared weights – while for many applications, rare words are as important (or mote!) than frequent words.</p>
<p>So one definite way to make training spend more time/effort, relatively, on lower-frequency words is to use a <em>more-aggressive</em> <code>sample</code> value, which means a <em>smaller</em> number, and more of the most-frequent-words being randomly skipped.</p>
<p>The default is <code>1e-04</code>; especially as your corpus grows, you could try a 10x smaller value like <code>1e-05</code>, a 100x smaller value like <code>1e-06</code>, or try even lower. As with other parameter tweaks, you should have some repeatable evaluation of the final vector quality, for your project purposes, that can be used to guide such adjustments.</p>
<p>A more aggressive <code>sample</code> can sometimes deliver a double-whammy of both faster training – by dropping the redundant high-frequency words – &amp; better final results – by both giving more weight to rarer words, &amp; effectively *shrinking• the context-windows whereever frequent-words are dropped. (The words are elided before the context-windows considered – so will move retained words that were just outside the window into it.)</p>
<p>I've seen a very-aggressive <code>window</code> value of <code>1e-06</code> or higher discard a majority of the pre-downsampling corpus, in typical natural language distributions. The saved training time might also then make it thinkable to consider otherwise impractically-larger values for other parameters which tend to increase training time (like <code>epochs</code>, <code>size</code>, <code>negative</code>, <code>window</code>).</p>
<p>There's another parameter, controlling the rates of negative-example sampling, that I believe is called <em>alpha</em> in the original word2vec paper, and frozen at <code>0.75</code> in the original Google <code>word2vec.c</code> tool. However, <a href="https://arxiv.org/abs/1804.04212" rel="nofollow noreferrer">some research</a> has suggested other values of this parameter may be useful in some applications – perhaps especially recommendation systems, and systems where the word-tokens don't have usual natural-language Zipfian distributions.</p>
<p>So, you may also want to try tinkering with that parameter. (Other implementations of word2vec, like Python Gensim's version, <a href="https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.Word2Vec" rel="nofollow noreferrer">offer this as a parameter <code>ns_exponent</code></a>.)</p>
<p>(Tinkering with the other parameters might help in your project aims, vis-a-vis the quality of less-frequent words' vectors, but not in an obvious way - you'd have to find such interactions by experiments in your domain.)</p>
word2vec: optimize for low-frequency words
<p>Given the classical implementation of <code>word2vec</code> by ‪Tomas Mikolov‬, what set of parameters (<code>window</code>, <code>sample</code>, <code>negative</code>, maybe <code>cbow</code>)</p>
<pre><code>./word2vec -train corpus.txt \
    -output vec.txt \
    -min-count 5 -size 150 \
    -window 5 -sample 1e-5 -negative 10
    -threads 16
</code></pre>
<p>optimize for computing better embeddings for low-frequency words (say with frequency 5 to 25)?</p>