2019)“[MT training] may incur an association of gender-speciﬁed pronouns (in the target) and gender-neutral ones (in the source) for
lexicon pairs that frequently collocate in the corpora. We claim
that this kind of phenomenon seriously threatens the fairness of a
translation system, in the sense that it lacks generality andinserts
social bias to the inference. Moreover, the input is not fully correct(considering gender-neutrality) and might offend the users
who expect fairer representations. ”Questionable
correlations,
other
representational
harmsQuestionable
correlations
Machine
translation
(Stanovsky
et al., 2019)“Learned models exhibit social bias when their training data
encode stereotypes not relevant for the task, but the correlations
are picked up anyway . ”Stereotyping,
questionable
correlationsStereotyping,
other
representational
harms (system
performance
differences),
questionable
correlations
Type-level
embeddings
(Zhao et al.,
2018b)“However, embeddings trained on human-generated corpora have
been demonstrated to inherit strong gender stereotypes that reﬂect social constructs....Such a bias substantially affects downstream applications ....This concerns the practitioners who use
the embedding model to build gender-sensitive applications such
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach