features associated with AAE than on text without
these features (Jørgensen et al., 2015, 2016; Blodgett et al., 2016, 2018), and that toxicity detection
systems score tweets containing features associated
with AAE as more offensive than tweets without them (Davidson et al., 2019; Sap et al., 2019).
These papers have been critical for highlighting
AAE as a language variety for which existing NLP
4Alsohttps://participatoryml.github.io/
5This language variety has had many different names
over the years, but is now generally called AfricanAmerican English (AAE), African-American Vernacular English (AA VE), or African-American Language (AAL) (Green,
2002; Wolfram and Schilling, 2015; Rickford and King, 2016).systems may not work, illustrating their limitations.
However, they do not conceptualize “racial bias” in
the same way. The ﬁrst four of these papers simply
focus on system performance differences between
text containing features associated with AAE and
text without these features. In contrast, the last
two papers also focus on such system performance
differences, but motivate this focus with the following additional reasoning: If tweets containing
features associated with AAE are scored as more
offensive than tweets without these features, then
this might (a) yield negative perceptions of AAE;
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach