title: Large Language Models as Zero-shot Dialogue State Tracker through
    Function Calling
summary: Large language models (LLMs) are increasingly prevalent in conversational
  systems due to their advanced understanding and generative capabilities in
  general contexts. However, their effectiveness in task-oriented dialogues
  (TOD), which requires not only response generation but also effective dialogue
  state tracking (DST) within specific tasks and domains, remains less
  satisfying. In this work, we propose a novel approach FnCTOD for solving DST
  with LLMs through function calling. This method improves zero-shot DST,
  allowing adaptation to diverse domains without extensive data collection or
  model tuning. Our experimental results demonstrate that our approach achieves
  exceptional performance with both modestly sized open-source and also
  proprietary LLMs: with in-context prompting it enables various 7B or 13B
  parameter models to surpass the previous state-of-the-art (SOTA) achieved by
  ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% Avg. JGA.
  Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%,
  respectively. We also show that by fine-tuning on a small collection of diverse
  task-oriented dialogues, we can equip modestly sized models, specifically a 13B
  parameter LLaMA2-Chat model, with function-calling capabilities and DST
  performance comparable to ChatGPT while maintaining their chat capabilities. We
  plan to open-source experimental code and model.
paper_authors._id: 65d2cd36351255ba48d4c127
paper_authors.name: Zekun Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:25:21.156Z
paper_authors._id: 61b312280d381ede9f1cf0c2
paper_authors.avatarUrl: /avatars/0427bfe4336fc5b08185590fd6675ee0.svg
paper_authors.fullname: Zekun Li
paper_authors.type: user
paper_authors.user: Zekunli
paper_authors._id: 65d2cd36351255ba48d4c128
paper_authors.name: Zhiyu Zoey Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:26:24.682Z
paper_authors._id: 6036333b1f06641e460cf5f9
paper_authors.avatarUrl: /avatars/25d47d311ee21c4b778edafef9c55172.svg
paper_authors.fullname: Zhiyu Chen
paper_authors.type: user
paper_authors.user: czyssrs
paper_authors._id: 65d2cd36351255ba48d4c129
paper_authors.name: Mike Ross
paper_authors._id: 65d2cd36351255ba48d4c12a
paper_authors.name: Patrick Huber
paper_authors._id: 65d2cd36351255ba48d4c12b
paper_authors.name: Seungwhan Moon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:27:20.872Z
paper_authors._id: 64c9ae24e761f47061448bfb
paper_authors.avatarUrl: /avatars/8835bcf769fd738e8c66fa7a52ddc421.svg
paper_authors.fullname: Seungwhan Moon
paper_authors.type: user
paper_authors.user: shanemoon27
paper_authors._id: 65d2cd36351255ba48d4c12c
paper_authors.name: Zhaojiang Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:28:08.732Z
paper_authors._id: 64b6d095dbbd1f2cdb62a946
paper_authors.avatarUrl: /avatars/b5f97735045d05ba5d30ee1fbf3f9bae.svg
paper_authors.fullname: Zhaojiang Lin
paper_authors.type: user
paper_authors.user: zlinao
paper_authors._id: 65d2cd36351255ba48d4c12d
paper_authors.name: Xin Luna Dong
paper_authors._id: 65d2cd36351255ba48d4c12e
paper_authors.name: Adithya Sagar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:28:32.578Z
paper_authors._id: 6313e3bb17838d05195197ab
paper_authors.avatarUrl: /avatars/6369a4b1d89597043f18eb9eb0735318.svg
paper_authors.fullname: Adithya Sagar
paper_authors.type: user
paper_authors.user: Adi007
paper_authors._id: 65d2cd36351255ba48d4c12f
paper_authors.name: Xifeng Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:28:38.825Z
paper_authors._id: 65cd4785c40ab294321d610e
paper_authors.avatarUrl: /avatars/3f0053aa2b3d90a10b60ab24cf575fd5.svg
paper_authors.fullname: Xifeng Yan
paper_authors.type: user
paper_authors.user: windwest
paper_authors._id: 65d2cd36351255ba48d4c130
paper_authors.name: Paul A. Crook
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T11:28:52.425Z
paper_authors._id: 64bee7581d40292dd31875d8
paper_authors.avatarUrl: /avatars/80329f5cad2c473fd42facc6226f77c5.svg
paper_authors.fullname: Paul Crook
paper_authors.type: user
paper_authors.user: pacrook
paper_title: Large Language Models as Zero-shot Dialogue State Tracker through
    Function Calling
paper_summary: Large language models (LLMs) are increasingly prevalent in conversational
  systems due to their advanced understanding and generative capabilities in
  general contexts. However, their effectiveness in task-oriented dialogues
  (TOD), which requires not only response generation but also effective dialogue
  state tracking (DST) within specific tasks and domains, remains less
  satisfying. In this work, we propose a novel approach FnCTOD for solving DST
  with LLMs through function calling. This method improves zero-shot DST,
  allowing adaptation to diverse domains without extensive data collection or
  model tuning. Our experimental results demonstrate that our approach achieves
  exceptional performance with both modestly sized open-source and also
  proprietary LLMs: with in-context prompting it enables various 7B or 13B
  parameter models to surpass the previous state-of-the-art (SOTA) achieved by
  ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% Avg. JGA.
  Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%,
  respectively. We also show that by fine-tuning on a small collection of diverse
  task-oriented dialogues, we can equip modestly sized models, specifically a 13B
  parameter LLaMA2-Chat model, with function-calling capabilities and DST
  performance comparable to ChatGPT while maintaining their chat capabilities. We
  plan to open-source experimental code and model.
paper_ai_summary: A novel function calling approach for dialogue state tracking using large language models enhances zero-shot and fine-tuned performance in task-oriented dialogues.
paper_ai_keywords: LLMs conversational systems task-oriented dialogues (TOD) dialogue state tracking (DST) function calling zero-shot DST open-source proprietary LLMs in-context prompting GPT-3.5 GPT-4 fine-tuning LLaMA2-Chat Avg. JGA