The main weakness of this paper is that it is not sufficiently grounded in the sociolinguistic side of bias measurement, and therefore makes some significant claims and design decisions that are not entirely appropriate.
The proposed task and the experimental design are predicated on the assumption that word senses as semantically disjoint with respect to social biases, but this is not really true (which is actually nodded to in the paper's discussion section). References to nationality and language can often be used metonymically to imply one another (particularly for voicing negative opinions), and colour and race are most certainly strongly correlated in visual design and storytelling (eg the use of dark colours, especially black, for evil in Western European storytelling, which is less common in African stories). Occupations and associated actions, while not used metonymically, may still be correlated: it is likely higher surprisal to refer to a woman "engineering a solution" than a man, for example. ( It is worth noting that the maximum pairwise similarity approach used in WAT calculation runs counter to this assumption of disjoint senses.)
There is also an important distinction between observability of a bias (e.g., through surveying/prompting native speakers) and measuring it using computational tools. Social biases may be observable without being measured by current approaches. These nuances are important to discuss throughout the paper: social bias assessment is not a computational task alone, but must incorporate human judgment and be framed through specific human viewpoints with acknowledged limitations.
There are several further issues with the precision of the language used in the paper that are important to address when discussing a sensitive topic such as social bias measurement.
- The term "bias" itself needs some context. Biases may be positive or neutral (eg preferring to complete "they sat at the computer to write ___" with "code" vs "a book" when trained on different data), as well as negative. As such, it is not clear what it means to have an "unbiased" model, or when that is desirable. A clearer definition of what kinds of bias are being assessed, what their impact is, and what mitigation of these biases might look like will help set the context better.
- The term "stereotype" is not used correctly. A stereotype is a commonly-held association between one group and some attribute or behavior; although most stereotypes involve negative attributes or disfavored behaviors, a stereotype is not inherently negative. The labels of "stereotype/anti-stereotype" are therefore not appropriate; positive/negative would be better. ( An anti-stereotype would be something that contradicts the stereotypical attribute or behavior.) Stereotypes are also socially-derived based on common beliefs/associations; for example, "Japanese people are stupid" is not a commonly-held belief, so this is not a stereotype.
- The terms "pleasant" and "unpleasant" are used, but are not clearly defined and feel subjective enough to mask the import of the difference being discussed. " Positive" and "negative" are the standard words used to discuss emotional valence of this type; these should either be used or the use of alternatives should be justified and clearly defined.
- Race and ethnicity are distinct constructs; the evaluation described in Section 4.2 is comparing race senses with colour senses, not ethnicity. ( Ethnicity is related more to heritage and origin, while race is a social group that one can be assigned to by others or identify with voluntarily.)
- Referring to nationality groups as "disadvantaged groups" (Line 286) doesn't quite work; while the racial identities that nationalities are correlated with may be disadvantaged, non-US nationalities themselves are not disadvantaged. ( Eg Scandinavian nationalities are very unlikely to be considered disadvantaged.)
It is not clear what the impact could be of measuring bias in sense embeddings or of debiasing sense embeddings. These models are much more rarely used than word embeddings, and in much more specialized settings.
Other issues with the presentation: - Section 3 is difficult to follow. It is not clear how (X and Y) and (A and B) are related, or what a high or low score on the s and w functions means.
- Figures 2 and 3 are unreadable.
- P-value calculation is discussed in Section 3, but no p-values are reported in the experimental results.
- The inclusion of the verb sense of "carpenter" in Section 4.3 is a little questionable; this is a very rarely-used sense. The other occupation/action words are commonly used and could reasonably be measured, but very few texts (for embedding training) will use carpenter as a verb, and many native speakers will not recognize it as correct.
- There is growing consensus to capitalize racial identifiers (certainly "Black", increasingly "White").
- Lines 442-445: This actually does not accurately simulate the word-level embedding case, as it assigns equal weight to all senses. In practice, word senses are more likely to be Zipf-distributed, so a word-level embedding model will be exposed to much more training data for some senses than for others.
- It is not clear how the categories in WEAT (Table 2) are associated with the social biases this paper is framed around.
- It would be helpful to mark the extended WEAT/WAT contributed by this paper using a modified label (eg WEAT*), in Table 2 and in text.
- The Ethical Statement is well written, but should be extended with some more concrete discussion of challenges not addressed in the dataset (eg gender beyond the binary, races other than Black, other sources of social bias).
- It would be helpful to have a listing of nationalities/racial identities/occupations included in the dataset (along with the adjectives used) as part of the paper, such as in supplementary tables.
- It would be interesting to see what human behavior is for these prompts/comparisons... - Dev et al 2019 and Nadeem et al 2020 references are missing publication information.
Typos - Figure 1, graph titles "embembeddings" - Line 051: extra "is" - Line 122: missing "are" - Line 603: "bises" -> "biases" - Line 610: missing "being" - Spaces not needed between section symbol and number - Spaces after non-terminal periods ("vs.", "cf.") should be escaped to avoid spacing issues
- It would be interesting to see what human behavior is for these prompts/comparisons...