list of the papers that fall into each category in the
appendix.) Again, we note that the counts do not
sum to 146, because some papers state multiple
motivations, propose multiple techniques, or propose a single technique for measuring or mitigating
multiple harms. Table 3, which is in the appendix,
contains examples of the papers’ motivations and
techniques across a range of different NLP tasks.
3 Findings
Categorizing the 146 papers’ motivations and proposed quantitative techniques for measuring or mitigating “bias” into the six categories listed above enabled us to identify several commonalities, which
we present below, along with illustrative quotes.
2We grouped several types of representational harms into
two categories to reﬂect that the main point of differentiation
between the 146 papers’ motivations and proposed quantitative
techniques for measuring or mitigating “bias” is whether or not
they focus on stereotyping. Among the papers that do not focus on stereotyping, we found that most lack sufﬁciently clear
motivations and techniques to reliably categorize them further.3.1 Motivations
Papers state a wide range of motivations,
multiple motivations, vague motivations, and
sometimes no motivations at all. We found that
the papers’ motivations span all six categories, with
several papers falling into each one. Appropriately,
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach