<p>You can use <a href="https://arxiv.org/pdf/1607.06520.pdf" rel="nofollow noreferrer">this paper</a> (Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings) method:</p>
<blockquote>
<p>To more robustly estimate bias, we shall aggregate
across multiple paired comparisons. By combining several directions, such as
<a href="https://i.stack.imgur.com/CibO3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/CibO3.png" alt="enter image description here" /></a></p>
<p>identify a gender direction <code>g ∈ R^d</code> that largely captures gender in the embedding. This direction helps us to quantify direct and indirect biases in words and associations.</p>
</blockquote>
<p>So, First, generating some pre-defined pair-differences utilized to generate a gender subspace. Then:</p>
<blockquote>
<p>To identify the gender subspace, we took the ten gender pair difference vectors and computed its principal
components (PCs). As Figure 6 shows, there is a single direction that explains the majority of variance
in these vectors. The first eigenvalue is significantly larger than the rest. Note that, from the randomness
in a finite sample of ten noisy vectors, one expects a decrease in eigenvalues. However, as also illustrated
in 6, the decrease one observes due to random sampling is much more gradual and uniform. Therefore we
hypothesize that the top PC, denoted by the unit vector g, captures the gender subspace. In general, the
gender subspace could be higher dimensional and all of our analysis and algorithms (described below) work
with general subspaces.</p>
<p><a href="https://i.stack.imgur.com/NMxnq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NMxnq.png" alt="enter image description here" /></a></p>
</blockquote>
<p>The generated vector <code>g</code> from PC analysis of the generated gender subspace hows the gender bias direction. Finally, to measure the <strong>DirectBias</strong>:</p>
<blockquote>
<p>To measure direct bias, we first identify words that should be gender-neutral for the application in question.
How to generate this set of gender-neutral words is described in Section 7. Given the gender neutral words,
denoted by N, and the gender direction learned from above, g, we define the direct gender bias of an embedding to be:
<a href="https://i.stack.imgur.com/CaVVj.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/CaVVj.png" alt="enter image description here" /></a></p>
</blockquote>
How to find gender bias in word embeddings?
<p>I have <code>glove.twitter.27B.200d.txt</code> word embeddings. These embeddings in <code>GloVe</code> format. I transfered it to <code>w2v</code> format using this code:</p>
<pre><code>model = KeyedVectors.load_word2vec_format(
&quot;data/glove.twitter.27B.200d.w2v.txt&quot;, binary=False
)
</code></pre>
<p><code>len(model.vocab) == 1193514</code></p>
<p>There is a gender bias in this word embeddings:</p>
<p><code>model.similarity(&quot;man&quot;, &quot;kitchen&quot;) == 0.32785824</code></p>
<p><code>model.similarity(&quot;woman&quot;, &quot;kitchen&quot;) == 0.40180725</code></p>
<p>I want to find a gender bias direction in this word embeddings, but not sure how.</p>