assume a strict gender dichotomy, thereby maintaining cisnormativity; Sap et al. (2019) focus on
the effect of priming annotators with information
about possible dialectal differences when asking
them to apply toxicity labels to sample tweets, ﬁnding that annotators who are primed are signiﬁcantly
less likely to label tweets containing features associated with African-American English as offensive.
4 A path forward
We now describe how researchers and practitioners
conducting work analyzing “bias” in NLP systems
might avoid the pitfalls presented in the previous
section—the beginnings of a path forward. We
propose three recommendations that should guide
such work, and, for each, provide several concrete
research questions. We emphasize that these questions are not comprehensive, and are intended to
generate further questions and lines of engagement.
Our three recommendations are as follows:
(R1) Ground work analyzing “bias” in NLP systems in the relevant literature outside of NLP
that explores the relationships between language and social hierarchies. Treat representational harms as harmful in their own right.
(R2) Provide explicit statements of why the
system behaviors that are described as “bias”
are harmful, in what ways, and to whom.
Be forthright about the normative reasoning
(Green, 2019) underlying these statements.
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach