title: Preference Learning Unlocks LLMs' Psycho-Counseling Skills
summary: Applying large language models (LLMs) to assist in psycho-counseling is an
  emerging and meaningful approach, driven by the significant gap between patient
  needs and the availability of mental health support. However, current LLMs
  struggle to consistently provide effective responses to client speeches,
  largely due to the lack of supervision from high-quality real psycho-counseling
  data, whose content is typically inaccessible due to client privacy concerns.
  Furthermore, the quality of therapists' responses in available sessions can
  vary significantly based on their professional training and experience.
  Assessing the quality of therapists' responses remains an open challenge. In
  this work, we address these challenges by first proposing a set of professional
  and comprehensive principles to evaluate therapists' responses to client
  speeches. Using these principles, we create a preference dataset,
  PsychoCounsel-Preference, which contains 36k high-quality preference comparison
  pairs. This dataset aligns with the preferences of professional
  psychotherapists, providing a robust foundation for evaluating and improving
  LLMs in psycho-counseling. Experiments on reward modeling and preference
  learning demonstrate that PsychoCounsel-Preference is an excellent resource for
  LLMs to acquire essential skills for responding to clients in a counseling
  session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an
  impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference,
  PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to
  facilitate the research of psycho-counseling with LLMs at:
  https://hf.co/Psychotherapy-LLM.
paper_authors._id: 67c36b35e12b50f698e7db1d
paper_authors.name: Mian Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:51:31.238Z
paper_authors._id: 650857fef3060ea840ffbbfe
paper_authors.avatarUrl: /avatars/3a339936021c040f19a21838ae1382c4.svg
paper_authors.fullname: Mian Zhang
paper_authors.type: user
paper_authors.user: billmianz
paper_authors._id: 67c36b35e12b50f698e7db1e
paper_authors.name: Shaun M. Eack
paper_authors._id: 67c36b35e12b50f698e7db1f
paper_authors.name: Zhiyu Zoey Chen
paper_title: Preference Learning Unlocks LLMs' Psycho-Counseling Skills
paper_summary: Applying large language models (LLMs) to assist in psycho-counseling is an
  emerging and meaningful approach, driven by the significant gap between patient
  needs and the availability of mental health support. However, current LLMs
  struggle to consistently provide effective responses to client speeches,
  largely due to the lack of supervision from high-quality real psycho-counseling
  data, whose content is typically inaccessible due to client privacy concerns.
  Furthermore, the quality of therapists' responses in available sessions can
  vary significantly based on their professional training and experience.
  Assessing the quality of therapists' responses remains an open challenge. In
  this work, we address these challenges by first proposing a set of professional
  and comprehensive principles to evaluate therapists' responses to client
  speeches. Using these principles, we create a preference dataset,
  PsychoCounsel-Preference, which contains 36k high-quality preference comparison
  pairs. This dataset aligns with the preferences of professional
  psychotherapists, providing a robust foundation for evaluating and improving
  LLMs in psycho-counseling. Experiments on reward modeling and preference
  learning demonstrate that PsychoCounsel-Preference is an excellent resource for
  LLMs to acquire essential skills for responding to clients in a counseling
  session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an
  impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference,
  PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to
  facilitate the research of psycho-counseling with LLMs at:
  https://hf.co/Psychotherapy-LLM.
paper_ai_summary: A preference dataset, PsychoCounsel-Preference, is created to evaluate and improve large language models for psycho-counseling, demonstrating significant advancements over existing models.
paper_ai_keywords: large language models psycho-counseling preference dataset preference comparison pairs professional psychotherapists reward modeling preference learning PsychoCounsel-Llama3-8B PsychoCounsel Llama3-8B-Reward