Elizabeth B.N. Sanders. 2002. From user-centered to
participatory design approaches. In Jorge Frascara,
editor, Design and the Social Sciences: Making Connections , pages 18–25. CRC Press.
Brenda Salenave Santana, Vinicius Woloszyn, and Leandro Krug Wives. 2018. Is there gender bias and
stereotype in Portuguese word embeddings? In
Proceedings of the International Conference on the
Computational Processing of Portuguese Student Research Workshop , Canela, Brazil.
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi,
and Noah A. Smith. 2019. The risk of racial bias in
hate speech detection. In Proceedings of the Association for Computational Linguistics (ACL) , pages
1668–1678, Florence, Italy.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social
Bias Frames: Reasoning about Social and Power Implications of Language. In Proceedings of the Association for Computational Linguistics (ACL) .
Hanna Sassaman, Jennifer Lee, Jenessa Irvine, and
Shankar Narayan. 2020. Creating CommunityBased Tech Policy: Case Studies, Lessons Learned,
and What Technologists and Communities Can Do
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach