or practices often translate into beliefs about their
speakers (e.g. Alim et al., 2016; Rosa and Flores,
2017; Craft et al., 2020). For example, in the U.S.,
the portrayal of non-white speakers’ language
varieties and practices as linguistically deﬁcient
helped to justify violent European colonialism, and
today continues to justify enduring racial hierarchies by maintaining views of non-white speakers
as lacking the language “required for complex
thinking processes and successful engagement
in the global economy” (Rosa and Flores, 2017).
Recognizing the role that language plays in
maintaining social hierarchies is critical to the
future of work analyzing “bias” in NLP systems.
First, it helps to explain why representational
harms are harmful in their own right. Second, the
complexity of the relationships between language
and social hierarchies illustrates why studying
“bias” in NLP systems is so challenging, suggesting
that researchers and practitioners will need to move
beyond existing algorithmic fairness techniques.
We argue that work must be grounded in the
relevant literature outside of NLP that examines
the relationships between language and social
hierarchies; without this grounding, researchers
and practitioners risk measuring or mitigating
only what is convenient to measure or mitigate,
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach