Natural Language Processing (NLP) systems learn harmful societal biases that cause them to widely proliferate inequality as they are deployed in more and more situations. To address and combat this , the NLP community relies on a variety of metrics to identify and quantify bias in black-box modelsand to guide efforts at debiasing . Some of these metrics are intrinsic, and are measured in word embedding spaces, and some are extrinsic, which measure the bias present downstream in the tasks that the word embeddings are plugged into. This research examines whether easy-to-measure intrinsic metrics correlate well to real world extrinsic metrics . We measure both intrinsic and extrinsic bias across hundreds of trained models covering different tasks and experimental conditions and find that there is no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We advise that efforts to debias embedding spaces be always also paired with measurement of downstream model bias, and suggest that that community increase effort into making downstream measurement more feasible via creation of additional challenge sets and annotated test data. We additionally release code, a new intrinsic metric, and an annotated test set for gender bias for hatespeech .
Natural Language Processing (NLP) systems learn harmful societal biases that cause them to amplify inequality as they are deployed in more and more situations. To guide efforts at debiasing these systems , the NLP community relies on a variety of metrics that quantify bias in models . Some of these metrics are intrinsic, measuring bias in word embedding spaces, and some are extrinsic, measuring bias in downstream tasks that the word embeddings enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare intrinsic and extrinsic metrics across hundreds of trained models covering different tasks and experimental conditions . Our results show no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We urge researchers working on debiasing to focus on extrinsic measures of bias, and to make using these measures more feasible via creation of new challenge sets and annotated test data. To aid this effort, we release code, a new intrinsic metric, and an annotated test set focused on gender bias in hate speech .
R
widely proliferate
amplify
R
address and combat this
guide efforts at debiasing these systems
R
to identify and
that
R
black-box modelsand to guide efforts at debiasing
models
R
and are measured
measuring bias
R
which measure the bias present downstream in the
measuring bias in downstream
R
are plugged into. This research examines whether easy-to-measure intrinsic metrics correlate well to real world extrinsic metrics . We measure both
enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare
R
bias
metrics
R
and find that there is
. Our results show
R
advise that efforts to debias embedding spaces be always also paired with measurement of downstream model
urge researchers working on debiasing to focus on extrinsic measures of
R
suggest that that community increase effort into making downstream measurement
to make using these measures
R
additional
new
R
We additionally
To aid this effort, we
R
for gender bias for hatespeech
focused on gender bias in hate speech