“gender-fair” language as an example, the hope
is that reducing asymmetries in language about
women and men will reduce asymmetries in their
social standing. Meanwhile, struggles over language use often arise from dominant social groups’
desire to “control both material and symbolic
resources”—i.e., “the right to decide what words
will mean and to control those meanings”—as was
the case in some white speakers’ insistence on
using offensive place names against the objections
of Indigenous speakers (Hill, 2008, Chapter 3).
Sociolinguists and linguistic anthropologists
have also examined language attitudes and language ideologies, or people’s metalinguistic beliefs
about language: Which language varieties or practices are taken as standard, ordinary, or unmarked?
Which are considered correct, prestigious, or appropriate for public use, and which are considered
incorrect, uneducated, or offensive (e.g., CampbellKibler, 2009; Preston, 2009; Loudermilk, 2015;
Lanehart and Malik, 2018)? Which are rendered invisible (Roche, 2019)?3Language ideologies play
a vital role in reinforcing and justifying social hierarchies because beliefs about language varieties
or practices often translate into beliefs about their
speakers (e.g. Alim et al., 2016; Rosa and Flores,
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach