title: CBT-Bench: Evaluating Large Language Models on Assisting Cognitive
    Behavior Therapy
summary: There is a significant gap between patient needs and available mental health
  support today. In this paper, we aim to thoroughly examine the potential of
  using Large Language Models (LLMs) to assist professional psychotherapy. To
  this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation
  of cognitive behavioral therapy (CBT) assistance. We include three levels of
  tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of
  multiple-choice questions; II: Cognitive model understanding, with the tasks of
  cognitive distortion classification, primary core belief classification, and
  fine-grained core belief classification; III: Therapeutic response generation,
  with the task of generating responses to patient speech in CBT therapy
  sessions. These tasks encompass key aspects of CBT that could potentially be
  enhanced through AI assistance, while also outlining a hierarchy of capability
  requirements, ranging from basic knowledge recitation to engaging in real
  therapeutic conversations. We evaluated representative LLMs on our benchmark.
  Experimental results indicate that while LLMs perform well in reciting CBT
  knowledge, they fall short in complex real-world scenarios requiring deep
  analysis of patients' cognitive structures and generating effective responses,
  suggesting potential future work.
paper_authors._id: 6712cece5fd735b90e62e60e
paper_authors.name: Mian Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:48:04.871Z
paper_authors._id: 650857fef3060ea840ffbbfe
paper_authors.avatarUrl: /avatars/3a339936021c040f19a21838ae1382c4.svg
paper_authors.fullname: Mian Zhang
paper_authors.type: user
paper_authors.user: billmianz
paper_authors._id: 6712cece5fd735b90e62e60f
paper_authors.name: Xianjun Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-22T09:40:13.798Z
paper_authors._id: 604137e5c15e823a685f2af2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/604137e5c15e823a685f2af2/rrxZUyrbLpKqCmG6xtP8v.jpeg
paper_authors.fullname: Xianjun Yang
paper_authors.type: user
paper_authors.user: Xianjun
paper_authors._id: 6712cece5fd735b90e62e610
paper_authors.name: Xinlu Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-22T09:40:32.686Z
paper_authors._id: 650b78532ec41a31f31163e9
paper_authors.avatarUrl: /avatars/522549da5ee5703b41396ce0b6811e89.svg
paper_authors.fullname: xinlu zhang
paper_authors.type: user
paper_authors.user: xz97
paper_authors._id: 6712cece5fd735b90e62e611
paper_authors.name: Travis Labrum
paper_authors._id: 6712cece5fd735b90e62e612
paper_authors.name: Jamie C. Chiu
paper_authors._id: 6712cece5fd735b90e62e613
paper_authors.name: Shaun M. Eack
paper_authors._id: 6712cece5fd735b90e62e614
paper_authors.name: Fei Fang
paper_authors._id: 6712cece5fd735b90e62e615
paper_authors.name: William Yang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-22T09:41:04.582Z
paper_authors._id: 6476607efb22e3b77f3f1452
paper_authors.avatarUrl: /avatars/22f40c81e6b6e09bdb66a5afd978f98b.svg
paper_authors.fullname: William Yang Wang
paper_authors.type: user
paper_authors.user: wangwilliamyang
paper_authors._id: 6712cece5fd735b90e62e616
paper_authors.name: Zhiyu Zoey Chen
paper_title: CBT-Bench: Evaluating Large Language Models on Assisting Cognitive
    Behavior Therapy
paper_summary: There is a significant gap between patient needs and available mental health
  support today. In this paper, we aim to thoroughly examine the potential of
  using Large Language Models (LLMs) to assist professional psychotherapy. To
  this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation
  of cognitive behavioral therapy (CBT) assistance. We include three levels of
  tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of
  multiple-choice questions; II: Cognitive model understanding, with the tasks of
  cognitive distortion classification, primary core belief classification, and
  fine-grained core belief classification; III: Therapeutic response generation,
  with the task of generating responses to patient speech in CBT therapy
  sessions. These tasks encompass key aspects of CBT that could potentially be
  enhanced through AI assistance, while also outlining a hierarchy of capability
  requirements, ranging from basic knowledge recitation to engaging in real
  therapeutic conversations. We evaluated representative LLMs on our benchmark.
  Experimental results indicate that while LLMs perform well in reciting CBT
  knowledge, they fall short in complex real-world scenarios requiring deep
  analysis of patients' cognitive structures and generating effective responses,
  suggesting potential future work.
paper_ai_keywords: Large Language Models (LLMs) CBT-BENCH cognitive behavioral therapy (CBT) multiple-choice questions cognitive distortion classification primary core belief classification fine-grained core belief classification therapeutic response generation CBT therapy sessions cognitive structures