title: Understanding and Predicting Derailment in Toxic Conversations on GitHub
summary: Software projects thrive on the involvement and contributions of individuals
  from different backgrounds. However, toxic language and negative interactions
  can hinder the participation and retention of contributors and alienate
  newcomers. Proactive moderation strategies aim to prevent toxicity from
  occurring by addressing conversations that have derailed from their intended
  purpose. This study aims to understand and predict conversational derailment
  leading to toxicity on GitHub.
    To facilitate this research, we curate a novel dataset comprising 202 toxic
  conversations from GitHub with annotated derailment points, along with 696
  non-toxic conversations as a baseline. Based on this dataset, we identify
  unique characteristics of toxic conversations and derailment points, including
  linguistic markers such as second-person pronouns, negation terms, and tones of
  Bitter Frustration and Impatience, as well as patterns in conversational
  dynamics between project contributors and external participants.
    Leveraging these empirical observations, we propose a proactive moderation
  approach to automatically detect and address potentially harmful conversations
  before escalation. By utilizing modern LLMs, we develop a conversation
  trajectory summary technique that captures the evolution of discussions and
  identifies early signs of derailment. Our experiments demonstrate that LLM
  prompts tailored to provide summaries of GitHub conversations achieve 69%
  F1-Score in predicting conversational derailment, strongly improving over a set
  of baseline approaches.
paper_authors._id: 67ca7fe72a83a60adcb6611a
paper_authors.name: Mia Mohammad Imran
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-07T09:09:36.214Z
paper_authors._id: 6331c3f618711776b468e9ec
paper_authors.avatarUrl: /avatars/af2c4bba031e474bf4fd2ea19e415aaf.svg
paper_authors.fullname: Mia Mohammad Imran
paper_authors.type: user
paper_authors.user: imranraad
paper_authors._id: 67ca7fe72a83a60adcb6611b
paper_authors.name: Robert Zita
paper_authors._id: 67ca7fe72a83a60adcb6611c
paper_authors.name: Rebekah Copeland
paper_authors._id: 67ca7fe72a83a60adcb6611d
paper_authors.name: Preetha Chatterjee
paper_authors._id: 67ca7fe72a83a60adcb6611e
paper_authors.name: Rahat Rizvi Rahman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T08:03:23.230Z
paper_authors._id: 6543e80aee7bbb5952983f05
paper_authors.avatarUrl: /avatars/fecd399f06dd0cafd9994fb8e26caeba.svg
paper_authors.fullname: Rahat Rizvi Rahman
paper_authors.type: user
paper_authors.user: rahat01
paper_authors._id: 67ca7fe72a83a60adcb6611f
paper_authors.name: Kostadin Damevski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:48:25.771Z
paper_authors._id: 64ca97e1d469fc2cf822d9f6
paper_authors.avatarUrl: /avatars/efec75e454ada7026e8497137de5bceb.svg
paper_authors.fullname: Kostadin Damevski
paper_authors.type: user
paper_authors.user: kdamevski
paper_title: Understanding and Predicting Derailment in Toxic Conversations on GitHub
paper_summary: Software projects thrive on the involvement and contributions of individuals
  from different backgrounds. However, toxic language and negative interactions
  can hinder the participation and retention of contributors and alienate
  newcomers. Proactive moderation strategies aim to prevent toxicity from
  occurring by addressing conversations that have derailed from their intended
  purpose. This study aims to understand and predict conversational derailment
  leading to toxicity on GitHub.
    To facilitate this research, we curate a novel dataset comprising 202 toxic
  conversations from GitHub with annotated derailment points, along with 696
  non-toxic conversations as a baseline. Based on this dataset, we identify
  unique characteristics of toxic conversations and derailment points, including
  linguistic markers such as second-person pronouns, negation terms, and tones of
  Bitter Frustration and Impatience, as well as patterns in conversational
  dynamics between project contributors and external participants.
    Leveraging these empirical observations, we propose a proactive moderation
  approach to automatically detect and address potentially harmful conversations
  before escalation. By utilizing modern LLMs, we develop a conversation
  trajectory summary technique that captures the evolution of discussions and
  identifies early signs of derailment. Our experiments demonstrate that LLM
  prompts tailored to provide summaries of GitHub conversations achieve 69%
  F1-Score in predicting conversational derailment, strongly improving over a set
  of baseline approaches.
paper_ai_summary: A proactive moderation system using LLMs predicts and addresses conversational derailment on GitHub to prevent toxicity.
paper_ai_keywords: LLMs conversation trajectory summary conversational derailment F1-Score