title: Moral Foundations of Large Language Models
summary: Moral foundations theory (MFT) is a psychological assessment tool that
  decomposes human moral reasoning into five factors, including care/harm,
  liberty/oppression, and sanctity/degradation (Graham et al., 2009). People vary
  in the weight they place on these dimensions when making moral decisions, in
  part due to their cultural upbringing and political ideology. As large language
  models (LLMs) are trained on datasets collected from the internet, they may
  reflect the biases that are present in such corpora. This paper uses MFT as a
  lens to analyze whether popular LLMs have acquired a bias towards a particular
  set of moral values. We analyze known LLMs and find they exhibit particular
  moral foundations, and show how these relate to human moral foundations and
  political affiliations. We also measure the consistency of these biases, or
  whether they vary strongly depending on the context of how the model is
  prompted. Finally, we show that we can adversarially select prompts that
  encourage the moral to exhibit a particular set of moral foundations, and that
  this can affect the model's behavior on downstream tasks. These findings help
  illustrate the potential risks and unintended consequences of LLMs assuming a
  particular moral stance.
paper_authors._id: 65388635954f6b0fd82d23d7
paper_authors.name: Marwa Abdulhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T11:54:04.779Z
paper_authors._id: 626af45dc804c432c1b3eb73
paper_authors.avatarUrl: /avatars/0a65ee91085e68c44489872f48fdba21.svg
paper_authors.fullname: Marwa Abdulhai
paper_authors.type: user
paper_authors.user: marwaa
paper_authors._id: 65388635954f6b0fd82d23d8
paper_authors.name: Gregory Serapio-Garcia
paper_authors._id: 65388635954f6b0fd82d23d9
paper_authors.name: Clément Crepy
paper_authors._id: 65388635954f6b0fd82d23da
paper_authors.name: Daria Valter
paper_authors._id: 65388635954f6b0fd82d23db
paper_authors.name: John Canny
paper_authors._id: 65388635954f6b0fd82d23dc
paper_authors.name: Natasha Jaques
paper_title: Moral Foundations of Large Language Models
paper_summary: Moral foundations theory (MFT) is a psychological assessment tool that
  decomposes human moral reasoning into five factors, including care/harm,
  liberty/oppression, and sanctity/degradation (Graham et al., 2009). People vary
  in the weight they place on these dimensions when making moral decisions, in
  part due to their cultural upbringing and political ideology. As large language
  models (LLMs) are trained on datasets collected from the internet, they may
  reflect the biases that are present in such corpora. This paper uses MFT as a
  lens to analyze whether popular LLMs have acquired a bias towards a particular
  set of moral values. We analyze known LLMs and find they exhibit particular
  moral foundations, and show how these relate to human moral foundations and
  political affiliations. We also measure the consistency of these biases, or
  whether they vary strongly depending on the context of how the model is
  prompted. Finally, we show that we can adversarially select prompts that
  encourage the moral to exhibit a particular set of moral foundations, and that
  this can affect the model's behavior on downstream tasks. These findings help
  illustrate the potential risks and unintended consequences of LLMs assuming a
  particular moral stance.
paper_ai_summary: Analysis of large language models using moral foundations theory reveals their biases in moral values and political affiliations, and demonstrates how these biases can influence model behavior in downstream tasks.
paper_ai_keywords: moral foundations theory denoise/harm liberty/oppression sanctity/degradation large language models moral biases political affiliations adversarial prompts downstream tasks