and communities affected by NLP systems. We
also provide several concrete research questions
that are implied by each of our recommendations.
2 Method
Our survey includes all papers known to us
analyzing “bias” in NLP systems—146 papers in
total. We omitted papers about speech, restricting
our survey to papers about written text only. To
identify the 146 papers, we ﬁrst searched the ACL
Anthology1for all papers with the keywords “bias”
or “fairness” that were made available prior to May
2020. We retained all papers about social “bias,”
and discarded all papers about other deﬁnitions of
the keywords (e.g., hypothesis-only bias, inductive
bias, media bias). We also discarded all papers using “bias” in NLP systems to measure social “bias”
in text or the real world (e.g., Garg et al., 2018).
To ensure that we did not exclude any relevant
1https://www.aclweb.org/anthology/NLP task Papers
Embeddings (type-level or contextualized) 54
Coreference resolution 20
Language modeling or dialogue generation 17
Hate-speech detection 17
Sentiment analysis 15
Machine translation 8
Tagging or parsing 5
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach