Prompt and Prejudice
<|reference_start|>Prompt and Prejudice: This paper investigates the impact of using first names in Large Language Models (LLMs) and Vision Language Models (VLMs), particularly when prompted with ethical decision-making tasks. We propose an approach that appends first names to ethically annotated text scenarios to reveal demographic biases in model outputs. Our study involves a curated list of more than 300 names representing diverse genders and ethnic backgrounds, tested across thousands of moral scenarios. Following the auditing methodologies from social sciences we propose a detailed analysis involving popular LLMs/VLMs to contribute to the field of responsible AI by emphasizing the importance of recognizing and mitigating biases in these systems. Furthermore, we introduce a novel benchmark, the Pratical Scenarios Benchmark (PSB), designed to assess the presence of biases involving gender or demographic prejudices in everyday decision-making scenarios as well as practical scenarios where an LLM might be used to make sensible decisions (e.g., granting mortgages or insurances). This benchmark allows for a comprehensive comparison of model behaviors across different demographic categories, highlighting the risks and biases that may arise in practical applications of LLMs and VLMs.<|reference_end|>
@article{berlincioni2024prompt,
    title={Prompt and Prejudice},
    author={Lorenzo Berlincioni, Luca Cultrera, Federico Becattini, Marco Bertini,
  Alberto Del Bimbo},
    journal={arXiv preprint arXiv:2408.04671},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.04671},
    primaryClass={cs.CL cs.AI cs.CY}
}