Recent improvements in large-scale language models have driven progress on automatic generation of syntactically and semantically consistent text for many real-world applications. Many of these advances leverage the availability of large corpora. While training on such corpora encourages the model to understand long-range dependencies in text, it can also result in the models internalizing the social biases present in the corpora. This paper aims to quantify and reduce biases exhibited by language models. Given a conditioning context (e.g. a writing prompt) and a language model, we analyze if (and how) the sentiment of the generated text is affected by changes in values of sensitive attributes (e.g. country names, occupations, genders, etc.) in the conditioning context, a.k.a. counterfactual evaluation. We quantify these biases by adapting individual and group fairness metrics from the fair machine learning literature. Extensive evaluation on two different corpora (news articles and Wikipedia) shows that state-of-the-art Transformer-based language models exhibit biases learned from data. We propose embedding-similarity and sentiment-similarity regularization methods that improve both individual and group fairness metrics without sacrificing perplexity and semantic similarity---a positive step toward development and deployment of fairer language models for real-world applications. Text representation learning methods (word and sentence encoders) trained on large unlabeled corpora are widely used in the development of natural language processing systems (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018; Devlin et al., 2018) . Progress in this area has led to consistent improvements of model performances on many downstream tasks. However, recent studies have found that both context-free and context-dependent word embedding models contain human-like semantic biases, including gender and race (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2019) . Zhao et al. (2018a) provide an insight into this phenomenon by showing that web corpora contain biases (e.g., gender) which are inherited by models trained on these datasets. In this work, we focus on language models which have been shown to exhibit systematic biases (Lu et al., 2018; Bordia & Bowman, 2019; Qian et al., 2019) . We train a Transformer-based language model (Vaswani et al., 2017; on two large corpora: Wikipedia articles from Wikitext-103 (Merity et al., 2016) and news articles from the English-language news corpus from . 1 We analyze systematic variations in sentiment scores of the text generated by the language model given a conditioning context, under different instantiations of control variables (e.g. country names, occupations, and person names) in the context. In a counterfactual experiment, we find that sentiment scores for the text generated by this language model vary substantially as we change the control variables in the context. We propose two approaches to reduce counterfactual sentiment biases based on the concept of embedding similarity or sentiment similarity. In the first method, we encourage hidden states of the conditioning context to be similar irrespective of the instantiations of the control variables in the context. In the second method, we regularize the difference between sentiment scores of various instantiations of the control variables. Experiments with counterfactual conditioning demonstrate that both of these methods reduce sentiment biases while retaining the generation capability of the language model, as measured by perplexity and semantic similarity. While specifying optimal model fairness behavior is difficult, our method provides a framework to address various fairness specifications and an important step toward the deployment of fairer language models. Our main contributions in this paper are: • We demonstrate systematic counterfactual sentiment biases in large-scale language models. • We present methods to quantify these biases by adopting individual and group fairness metrics from the fair machine learning literature. • We propose embedding and sentiment similarity-based methods for training language models to be invariant to certain transformations of their inputs. • We empirically demonstrate the efficacy of these methods to reduce counterfactual sentiment biases of language models. We use a sentiment classifier as a proxy to measure biases in this paper. We note that the classifier itself is not perfect and might exhibit some biases. We leave investigations of an unbiased evaluator to future work. As large-scale language models are increasingly deployed for real-world applications, developing methods for assessing and mitigating bias with respect to sensitive attributes may be an increasingly important area of inquiry for facilitating pro-social outcomes. Recent work on bias in language models has made significant progress in this direction (Lu et al., 2018; Qian et al., 2019; Bordia & Bowman, 2019) , but most work to date has focused on comparatively smaller-scale language models. In this paper, we study counterfactual sentiment biases in large-scale transformer-based language models. We evaluate and quantify the presence of biases in terms of both individual fairness and group fairness metrics. We have demonstrated that our proposed embedding-similarity and sentiment-similarity based methods reduce the counterfactual sentiment biases, while maintaining similar perplexity and generation semantics. While specifying optimal model fairness behavior is difficult, our method provides a framework to address various fairness specifications and an important step toward the deployment of fairer language models. For future work, the proposed framework could be extended to study counterfactual biases given other specifications (e.g. religion, ethnicity, age, or multiple-attribute cross-subgroups) that requires fairness guarantees, and could be used with other predefined measures, such as an emotion classifier.
We reduce sentiment biases based on counterfactual evaluation of text generation using language models. This paper measures sentiment bias in language models as reflected by text generated by the models, and adds other objective terms to the usual language modeling objective to reduce bias. This paper proposes to evaluate bias in pre-trained language models by using a fixed sentiment system and tests several different prefix templates. A method based on semantic simiilarity and a method based on sentiment similarity to debias the neural language models trained from large datasets.