DialogStudio: Towards Richest and Most Diverse Unified Dataset
  Collection for Conversational AI
Despite advancements in conversational AI, language models encounter
challenges to handle diverse conversational tasks, and existing dialogue
dataset collections often lack diversity and comprehensiveness. To tackle these
issues, we introduce DialogStudio: the largest and most diverse collection of
dialogue datasets, unified under a consistent format while preserving their
original information. Our collection encompasses data from open-domain
dialogues, task-oriented dialogues, natural language understanding,
conversational recommendation, dialogue summarization, and knowledge-grounded
dialogues, making it an incredibly rich and diverse resource for dialogue
research and model training. To further enhance the utility of DialogStudio, we
identify the licenses for each dataset and design domain-aware prompts for
selected dialogues to facilitate instruction-aware fine-tuning. Furthermore, we
develop conversational AI models using the dataset collection, and our
experiments in both zero-shot and few-shot learning scenarios demonstrate the
superiority of DialogStudio. To improve transparency and support dataset and
task-based research, as well as language model pre-training, all datasets,
licenses, codes, and models associated with DialogStudio are made publicly
accessible at https://github.com/salesforce/DialogStudio
64b89f79d6ced0fd74b332b5
Jianguo Zhang
admin_assigned
2023-07-20T07:40:27.185Z
630fcbd758d83e8f64d82777
https://cdn-avatars.huggingface.co/v1/production/uploads/630fcbd758d83e8f64d82777/0jwTa1MzeXZbDtJ-uXGo6.jpeg
Jianguo Zhang
user
jianguozhang
64b89f79d6ced0fd74b332b6
Kun Qian
admin_assigned
2023-07-20T07:40:53.596Z
646978ec188ca2e6bc5be679
/avatars/29e564ce2da89c6a9c59d1bdb9bb7d0b.svg
Kun Qian
user
qbetterk
64b89f79d6ced0fd74b332b7
Zhiwei Liu
claimed_verified
2023-07-25T08:08:15.249Z
64470130bd9e0e0edf5227d7
/avatars/89f118f880cce3d01658b123bbbf4402.svg
Zhiwei Liu
user
jimzhiwei
64b89f79d6ced0fd74b332b8
Shelby Heinecke
64b89f79d6ced0fd74b332b9
Rui Meng
claimed_verified
2023-07-21T07:36:17.823Z
61b022b39f13f47207d335cf
/avatars/9cf00a775b80230aa32ef68ea7377833.svg
Rui
user
memray
64b89f79d6ced0fd74b332ba
Ye Liu
64b89f79d6ced0fd74b332bb
Zhou Yu
64be465137b81dbb07682202
Huan Wang
claimed_verified
2023-07-25T08:08:05.346Z
64b96d0bea67d88ebec0f983
https://cdn-avatars.huggingface.co/v1/production/uploads/64b96d0bea67d88ebec0f983/SnZwF6_ABRXhU64CoFoZm.jpeg
Huan Wang
user
joyousprince
64b89f79d6ced0fd74b332bc
Silvio Savarese
64b89f79d6ced0fd74b332bd
Caiming Xiong
admin_assigned
2023-07-20T07:42:13.024Z
649dbcc4e0fff1ed099dc80a
/avatars/c87c273ca628dbcddccbf1ee19b2ce33.svg
Caiming Xiong
user
cxiong
DialogStudio: Towards Richest and Most Diverse Unified Dataset
  Collection for Conversational AI
Despite advancements in conversational AI, language models encounter
challenges to handle diverse conversational tasks, and existing dialogue
dataset collections often lack diversity and comprehensiveness. To tackle these
issues, we introduce DialogStudio: the largest and most diverse collection of
dialogue datasets, unified under a consistent format while preserving their
original information. Our collection encompasses data from open-domain
dialogues, task-oriented dialogues, natural language understanding,
conversational recommendation, dialogue summarization, and knowledge-grounded
dialogues, making it an incredibly rich and diverse resource for dialogue
research and model training. To further enhance the utility of DialogStudio, we
identify the licenses for each dataset and design domain-aware prompts for
selected dialogues to facilitate instruction-aware fine-tuning. Furthermore, we
develop conversational AI models using the dataset collection, and our
experiments in both zero-shot and few-shot learning scenarios demonstrate the
superiority of DialogStudio. To improve transparency and support dataset and
task-based research, as well as language model pre-training, all datasets,
licenses, codes, and models associated with DialogStudio are made publicly
accessible at https://github.com/salesforce/DialogStudio
DialogStudio is a comprehensive and diverse collection of dialogue datasets designed to address challenges in conversational AI and improve dialogue research and model training.
conversational AI dialogue datasets open-domain dialogues task-oriented dialogues natural language understanding conversational recommendation dialogue summarization knowledge-grounded dialogues instruction-aware fine-tuning zero-shot learning few-shot learning