title: SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for
    Enterprise Use
summary: Enterprise customers are increasingly adopting Large Language Models (LLMs)
  for critical communication tasks, such as drafting emails, crafting sales
  pitches, and composing casual messages. Deploying such models across different
  regions requires them to understand diverse cultural and linguistic contexts
  and generate safe and respectful responses. For enterprise applications, it is
  crucial to mitigate reputational risks, maintain trust, and ensure compliance
  by effectively identifying and handling unsafe or offensive language. To
  address this, we introduce SweEval, a benchmark simulating real-world scenarios
  with variations in tone (positive or negative) and context (formal or
  informal). The prompts explicitly instruct the model to include specific swear
  words while completing the task. This benchmark evaluates whether LLMs comply
  with or resist such inappropriate instructions and assesses their alignment
  with ethical frameworks, cultural nuances, and language comprehension
  capabilities. In order to advance research in building ethically aligned AI
  systems for enterprise use and beyond, we release the dataset and code:
  https://github.com/amitbcp/multilingual_profanity.
paper_authors._id: 68349ace56ca97d1b2f1eb7c
paper_authors.name: Hitesh Laxmichand Patel
paper_authors._id: 68349ace56ca97d1b2f1eb7d
paper_authors.name: Amit Agarwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:49:14.382Z
paper_authors._id: 6321a50a6b1992383fa00db1
paper_authors.avatarUrl: /avatars/92311b4a5e8bcc8f5477747621334fc8.svg
paper_authors.fullname: Amit Agarwal
paper_authors.type: user
paper_authors.user: amitbcp
paper_authors._id: 68349ace56ca97d1b2f1eb7e
paper_authors.name: Arion Das
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:53:29.445Z
paper_authors._id: 655c848048320febfc45db74
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/-An-36AfvExD_EfLFXHOP.jpeg
paper_authors.fullname: Arion Das
paper_authors.type: user
paper_authors.user: ariondas
paper_authors._id: 68349ace56ca97d1b2f1eb7f
paper_authors.name: Bhargava Kumar
paper_authors._id: 68349ace56ca97d1b2f1eb80
paper_authors.name: Srikant Panda
paper_authors._id: 68349ace56ca97d1b2f1eb81
paper_authors.name: Priyaranjan Pattnayak
paper_authors._id: 68349ace56ca97d1b2f1eb82
paper_authors.name: Taki Hasan Rafi
paper_authors._id: 68349ace56ca97d1b2f1eb83
paper_authors.name: Tejaswini Kumar
paper_authors._id: 68349ace56ca97d1b2f1eb84
paper_authors.name: Dong-Kyu Chae
paper_title: SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for
    Enterprise Use
paper_summary: Enterprise customers are increasingly adopting Large Language Models (LLMs)
  for critical communication tasks, such as drafting emails, crafting sales
  pitches, and composing casual messages. Deploying such models across different
  regions requires them to understand diverse cultural and linguistic contexts
  and generate safe and respectful responses. For enterprise applications, it is
  crucial to mitigate reputational risks, maintain trust, and ensure compliance
  by effectively identifying and handling unsafe or offensive language. To
  address this, we introduce SweEval, a benchmark simulating real-world scenarios
  with variations in tone (positive or negative) and context (formal or
  informal). The prompts explicitly instruct the model to include specific swear
  words while completing the task. This benchmark evaluates whether LLMs comply
  with or resist such inappropriate instructions and assesses their alignment
  with ethical frameworks, cultural nuances, and language comprehension
  capabilities. In order to advance research in building ethically aligned AI
  systems for enterprise use and beyond, we release the dataset and code:
  https://github.com/amitbcp/multilingual_profanity.
paper_ai_summary: SweEval is a benchmark for evaluating Large Language Models' compliance with ethical guidelines and cultural nuances when instructed to include offensive language.
paper_ai_keywords: Large Language Models LLMs SweEval benchmark ethical alignment cultural nuances language comprehension ethical frameworks