Large Language Models as Zero-shot Dialogue State Tracker through
  Function Calling
Large language models (LLMs) are increasingly prevalent in conversational
systems due to their advanced understanding and generative capabilities in
general contexts. However, their effectiveness in task-oriented dialogues
(TOD), which requires not only response generation but also effective dialogue
state tracking (DST) within specific tasks and domains, remains less
satisfying. In this work, we propose a novel approach FnCTOD for solving DST
with LLMs through function calling. This method improves zero-shot DST,
allowing adaptation to diverse domains without extensive data collection or
model tuning. Our experimental results demonstrate that our approach achieves
exceptional performance with both modestly sized open-source and also
proprietary LLMs: with in-context prompting it enables various 7B or 13B
parameter models to surpass the previous state-of-the-art (SOTA) achieved by
ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% Avg. JGA.
Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%,
respectively. We also show that by fine-tuning on a small collection of diverse
task-oriented dialogues, we can equip modestly sized models, specifically a 13B
parameter LLaMA2-Chat model, with function-calling capabilities and DST
performance comparable to ChatGPT while maintaining their chat capabilities. We
plan to open-source experimental code and model.
65d2cd36351255ba48d4c127
Zekun Li
admin_assigned
2024-02-19T11:25:21.156Z
61b312280d381ede9f1cf0c2
/avatars/0427bfe4336fc5b08185590fd6675ee0.svg
Zekun Li
user
Zekunli
65d2cd36351255ba48d4c128
Zhiyu Zoey Chen
admin_assigned
2024-02-19T11:26:24.682Z
6036333b1f06641e460cf5f9
/avatars/25d47d311ee21c4b778edafef9c55172.svg
Zhiyu Chen
user
czyssrs
65d2cd36351255ba48d4c129
Mike Ross
65d2cd36351255ba48d4c12a
Patrick Huber
65d2cd36351255ba48d4c12b
Seungwhan Moon
admin_assigned
2024-02-19T11:27:20.872Z
64c9ae24e761f47061448bfb
/avatars/8835bcf769fd738e8c66fa7a52ddc421.svg
Seungwhan Moon
user
shanemoon27
65d2cd36351255ba48d4c12c
Zhaojiang Lin
admin_assigned
2024-02-19T11:28:08.732Z
64b6d095dbbd1f2cdb62a946
/avatars/b5f97735045d05ba5d30ee1fbf3f9bae.svg
Zhaojiang Lin
user
zlinao
65d2cd36351255ba48d4c12d
Xin Luna Dong
65d2cd36351255ba48d4c12e
Adithya Sagar
admin_assigned
2024-02-19T11:28:32.578Z
6313e3bb17838d05195197ab
/avatars/6369a4b1d89597043f18eb9eb0735318.svg
Adithya Sagar
user
Adi007
65d2cd36351255ba48d4c12f
Xifeng Yan
admin_assigned
2024-02-19T11:28:38.825Z
65cd4785c40ab294321d610e
/avatars/3f0053aa2b3d90a10b60ab24cf575fd5.svg
Xifeng Yan
user
windwest
65d2cd36351255ba48d4c130
Paul A. Crook
admin_assigned
2024-02-19T11:28:52.425Z
64bee7581d40292dd31875d8
/avatars/80329f5cad2c473fd42facc6226f77c5.svg
Paul Crook
user
pacrook
Large Language Models as Zero-shot Dialogue State Tracker through
  Function Calling
Large language models (LLMs) are increasingly prevalent in conversational
systems due to their advanced understanding and generative capabilities in
general contexts. However, their effectiveness in task-oriented dialogues
(TOD), which requires not only response generation but also effective dialogue
state tracking (DST) within specific tasks and domains, remains less
satisfying. In this work, we propose a novel approach FnCTOD for solving DST
with LLMs through function calling. This method improves zero-shot DST,
allowing adaptation to diverse domains without extensive data collection or
model tuning. Our experimental results demonstrate that our approach achieves
exceptional performance with both modestly sized open-source and also
proprietary LLMs: with in-context prompting it enables various 7B or 13B
parameter models to surpass the previous state-of-the-art (SOTA) achieved by
ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% Avg. JGA.
Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%,
respectively. We also show that by fine-tuning on a small collection of diverse
task-oriented dialogues, we can equip modestly sized models, specifically a 13B
parameter LLaMA2-Chat model, with function-calling capabilities and DST
performance comparable to ChatGPT while maintaining their chat capabilities. We
plan to open-source experimental code and model.
A novel function calling approach for dialogue state tracking using large language models enhances zero-shot and fine-tuned performance in task-oriented dialogues.
LLMs conversational systems task-oriented dialogues (TOD) dialogue state tracking (DST) function calling zero-shot DST open-source proprietary LLMs in-context prompting GPT-3.5 GPT-4 fine-tuning LLaMA2-Chat Avg. JGA