Natural Language Processing (NLP) systems learn harmful societal biases that cause them to widely proliferate inequality as they are deployed in more and more situations. To address and combat this , the NLP community relies on a variety of metrics to identify and quantify bias in black-box modelsand to guide efforts at debiasing . Some of these metrics are intrinsic, and are measured in word embedding spaces, and some are extrinsic, which measure the bias present downstream in the tasks that the word embeddings are plugged into. This research examines whether easy-to-measure intrinsic metrics correlate well to real world extrinsic metrics . We measure both intrinsic and extrinsic bias across hundreds of trained models covering different tasks and experimental conditions and find that there is no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We advise that efforts to debias embedding spaces be always also paired with measurement of downstream model bias, and suggest that that community increase effort into making downstream measurement more feasible via creation of additional challenge sets and annotated test data. We additionally release code, a new intrinsic metric, and an annotated test set for gender bias for hatespeech .
Natural Language Processing (NLP) systems learn harmful societal biases that cause them to amplify inequality as they are deployed in more and more situations. To guide efforts at debiasing these systems , the NLP community relies on a variety of metrics that quantify bias in models . Some of these metrics are intrinsic, measuring bias in word embedding spaces, and some are extrinsic, measuring bias in downstream tasks that the word embeddings enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare intrinsic and extrinsic metrics across hundreds of trained models covering different tasks and experimental conditions . Our results show no reliable correlation between these metrics that holds in all scenarios across tasks and languages. We urge researchers working on debiasing to focus on extrinsic measures of bias, and to make using these measures more feasible via creation of new challenge sets and annotated test data. To aid this effort, we release code, a new intrinsic metric, and an annotated test set focused on gender bias in hate speech .
R
widely proliferate
amplify
clarity
clarity clarity clarity
R
address and combat this
guide efforts at debiasing these systems
clarity
meaning-changed clarity clarity
R
to identify and
that
fluency
clarity fluency fluency
R
black-box modelsand to guide efforts at debiasing
models
clarity
clarity clarity clarity
R
and are measured
measuring bias
clarity
clarity clarity clarity
R
which measure the bias present downstream in the
measuring bias in downstream
clarity
clarity clarity clarity
R
are plugged into. This research examines whether easy-to-measure intrinsic metrics correlate well to real world extrinsic metrics . We measure both
enable. Do these intrinsic and extrinsic metrics correlate with each other? We compare
clarity
clarity style clarity
R
bias
metrics
clarity
clarity clarity clarity
R
and find that there is
. Our results show
coherence
coherence style coherence
R
advise that efforts to debias embedding spaces be always also paired with measurement of downstream model
urge researchers working on debiasing to focus on extrinsic measures of
clarity
clarity clarity clarity
R
suggest that that community increase effort into making downstream measurement
to make using these measures
clarity
clarity clarity clarity
R
additional
new
clarity
meaning-changed clarity clarity
R
We additionally
To aid this effort, we
clarity
style clarity clarity
R
for gender bias for hatespeech
focused on gender bias in hate speech
clarity
clarity clarity fluency