ideologies determine what we mean by “bias” in
the ﬁrst place. As a starting point, we therefore
provide the following concrete research questions:
.What kinds of system behaviors are described
as “bias”? What are their potential sources (e.g.,
general assumptions, task deﬁnition, data)?
.In what ways are these system behaviors harmful, to whom are they harmful, and why?
.What are the social values (obvious or not) thatunderpin this conceptualization of “bias?”
4.3 Language use in practice
Finally, we turn to (R3). Our perspective, which
rests on a greater recognition of the relationships
between language and social hierarchies, suggests
several directions for examining language use in
practice. Here, we focus on two. First, because language is necessarily situated, and because different
social groups have different lived experiences due
to their different social positions (Hanna et al.,
2020)—particularly groups at the intersections
of multiple axes of oppression—we recommend
that researchers and practitioners center work
analyzing “bias” in NLP systems around the lived
experiences of members of communities affected
by these systems. Second, we recommend that
the power relations between technologists and
such communities be interrogated and reimagined.
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.
Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach