elements. We recorded all terms related to *speech that could be perceived as offensive, derogatory, or in any way harmful, and that is motivated, in whole or in a part, by someone’s bias against an aspect of a group of people, or related to commentary about such speech by others, or related to speech that aims to counter any type of speech that this definition covers.*

This expansion was independently done in two iterations for both Twitter and Reddit. First, one of the authors did an annotation pass to identify new query terms. Second, as we favored recall, at least one other author did an additional annotation pass over the terms rejected by the first annotator.

[**External lexicon.** ]{} To further expand our list of query terms, we added terms from a lexicon built using HateBase,[^5] a website that compiles phrases submitted and tagged by internet users as constituting hate speech. Given that only an estimated 5% of messages containing HateBase terms were actually identified as hateful; instead of directly using these terms, we used 163 unique terms extracted from Twitter messages containing HateBase terms and manually annotated as hateful or offensive by @davidson2017automated.[^6]

Data Acquisition
----------------

Table \[tbl:datasets\] presents 