Structure, History, and Use . Routledge.
Michael J. Muller. 2007. Participatory Design: The
Third Space in HCI. In The Human-Computer Interaction Handbook , pages 1087–1108. CRC Press.
Moin Nadeem, Anna Bethke, and Siva Reddy.
2020. StereoSet: Measuring stereotypical bias
in pretrained language models. arXiv preprint
arXiv:2004.09456 .
Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and
Theo Meder. 2013. “How Old Do You Think I
Am?”: A Study of Language and Age in Twitter. In
Proceedings of the Conference on Web and Social
Media (ICWSM) , pages 439–448, Boston, MA.
Malvina Nissim, Rik van Noord, and Rob van der Goot.
2020. Fair is better than sensational: Man is to doctor as woman is to doctor. Computational Linguistics.
Debora Nozza, Claudia V olpetti, and Elisabetta Fersini.
2019. Unintended Bias in Misogyny Detection. In
Proceedings of the Conference on Web Intelligence ,
pages 149–155.
Alexandra Olteanu, Carlos Castillo, Fernando Diaz,
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach