reasoning. Researchers and practitioners should
be concerned about “racial bias” in toxicity
detection systems not only because performance
differences impair system performance, but
because they reproduce longstanding injustices of
stigmatization and disenfranchisement for speakers
of AAE. In re-stigmatizing AAE, they reproduce
language ideologies in which AAE is viewed as
ungrammatical, uneducated, and offensive. These
ideologies, in turn, enable linguistic discrimination
and justify enduring racial hierarchies (Rosa and
Flores, 2017). Our perspective, which understands
racial hierarchies and raciolinguistic ideologies as
structural conditions that govern the development
and deployment of technology, implies that
techniques for measuring or mitigating “bias”
in NLP systems will necessarily be incomplete
unless they interrogate and dismantle these
structural conditions, including the power relations
between technologists and racialized communities.
We emphasize that engaging with the literature
on AAE, racial hierarchies in the U.S., and
raciolinguistic ideologies can generate new lines of
engagement. These lines include work on the ways
that the decisions made during the developmentand deployment of NLP systems produce stigmatization and disenfranchisement, and work on AAE
use in practice, such as the ways that speakers
of AAE interact with NLP systems that were not
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach