Certifying Counterfactual Bias in LLMs
Isha Chaudhary Qian Hu Manoj Kumar Morteza Ziyadi Rahul Gupta Gagandeep Singh
Large Language Models (LLMs) can produce biased responses that can cause representational harms. However, conventional studies are insufficient to thoroughly
evaluate biases across LLM responses for different demographic groups (a.k.a.
counterfactual bias), as they do not scale to large number of inputs and do not
provide guarantees. Therefore, we propose the first framework, LLMCert-B that
certifies LLMs for counterfactual bias on distributions of prompts. A certificate
consists of high-confidence bounds on the probability of unbiased LLM responses
for any set of counterfactual prompts - prompts differing by demographic groups,
sampled from a distribution. We illustrate counterfactual bias certification for
distributions of counterfactual prompts created by applying prefixes sampled from
prefix distributions, to a given set of prompts. We consider prefix distributions consisting random token sequences, mixtures of manual jailbreaks, and perturbations
of jailbreaks in LLM’s embedding space. We generate non-trivial certificates for
SOTA LLMs, exposing their vulnerabilities over distributions of prompts generated
from computationally inexpensive prefix distributions.
Large Language Models, Bias, Certification
We present the first framework to formally certify counterfactual bias in the responses of LLMs.