Recourse for reclamation: Chatting with generative language models
Researchers and developers increasingly rely on toxicity scoring to moderate
generative language model outputs, in settings such as customer service,
information retrieval, and content generation. However, toxicity scoring may
render pertinent information inaccessible, rigidify or "value-lock" cultural
norms, and prevent language reclamation processes, particularly for
marginalized people. In this work, we extend the concept of algorithmic
recourse to generative language models: we provide users a novel mechanism to
achieve their desired prediction by dynamically setting thresholds for toxicity
filtering. Users thereby exercise increased agency relative to interactions
with the baseline system. A pilot study (n = 30) supports the potential of
our proposed recourse mechanism, indicating improvements in usability compared
to fixed-threshold toxicity-filtering of model outputs. Future work should
explore the intersection of toxicity scoring, model controllability, user
agency, and language reclamation processes -- particularly with regard to the
bias that many communities encounter when interacting with generative language
models.
65fd02aabe8494c6be3f1fba
Jennifer Chien
claimed_verified
2024-03-22T11:49:59.119Z
65fd7039cce786577103dcfe
/avatars/7ebc5ec9d7851ca1a0d33cc0bc703f81.svg
Jennifer Chien
user
thechiennifer
65fd02aabe8494c6be3f1fbb
Kevin R. McKee
admin_assigned
2024-03-22T10:32:06.717Z
624911a1edf42bf849d0ffc5
/avatars/fdf70a412b2b90d0407dc6adb484319f.svg
Kevin Mckee
user
Kmckee95
65fd02aabe8494c6be3f1fbc
Jackie Kay
65fd02aabe8494c6be3f1fbd
William Isaac
Recourse for reclamation: Chatting with generative language models
Researchers and developers increasingly rely on toxicity scoring to moderate
generative language model outputs, in settings such as customer service,
information retrieval, and content generation. However, toxicity scoring may
render pertinent information inaccessible, rigidify or "value-lock" cultural
norms, and prevent language reclamation processes, particularly for
marginalized people. In this work, we extend the concept of algorithmic
recourse to generative language models: we provide users a novel mechanism to
achieve their desired prediction by dynamically setting thresholds for toxicity
filtering. Users thereby exercise increased agency relative to interactions
with the baseline system. A pilot study (n = 30) supports the potential of
our proposed recourse mechanism, indicating improvements in usability compared
to fixed-threshold toxicity-filtering of model outputs. Future work should
explore the intersection of toxicity scoring, model controllability, user
agency, and language reclamation processes -- particularly with regard to the
bias that many communities encounter when interacting with generative language
models.
A mechanism for dynamic toxicity thresholding in generative language models enhances user agency and usability compared to fixed thresholds.
generative language models toxicity scoring algorithmic recourse dynamic thresholds fixed-threshold toxicity-filtering user agency language reclamation bias