AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
As a part of an embodied agent, Large Language Models (LLMs) are typically
used for behavior planning given natural language instructions from the user.
However, dealing with ambiguous instructions in real-world environments remains
a challenge for LLMs. Various methods for task ambiguity detection have been
proposed. However, it is difficult to compare them because they are tested on
different datasets and there is no universal benchmark. For this reason, we
propose AmbiK (Ambiguous Tasks in Kitchen Environment), the fully textual
dataset of ambiguous instructions addressed to a robot in a kitchen
environment. AmbiK was collected with the assistance of LLMs and is
human-validated. It comprises 1000 pairs of ambiguous tasks and their
unambiguous counterparts, categorized by ambiguity type (Human Preferences,
Common Sense Knowledge, Safety), with environment descriptions, clarifying
questions and answers, user intents, and task plans, for a total of 2000 tasks.
We hope that AmbiK will enable researchers to perform a unified comparison of
ambiguity detection methods. AmbiK is available at
https://github.com/cog-model/AmbiK-dataset.
684153cf911d1b3135fa5dfe
Anastasiia Ivanova
claimed_verified
2025-06-24T08:11:08.331Z
64419b73bd0c972652999558
/avatars/cce94b2a4033e2b8803af085458bd7b9.svg
Anastasia Ivanova
user
IvAnastasia
684153cf911d1b3135fa5dff
Eva Bakaeva
claimed_verified
2025-06-05T10:00:02.435Z
661af24d8328f43c6abc2d11
/avatars/afe7eaf1f7a378dbcdba5cd3e86adf9c.svg
Eva
user
tenebrissilvam
684153cf911d1b3135fa5e00
Zoya Volovikova
extracted_pending
2025-06-05T08:22:39.926Z
64198f70ed725fef6442b37e
/avatars/580ab07a3067a9deb2977b0894226fe3.svg
Alexey Kovalev
user
AlexeyKov
684153cf911d1b3135fa5e01
Alexey K. Kovalev
684153cf911d1b3135fa5e02
Aleksandr I. Panov
AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
As a part of an embodied agent, Large Language Models (LLMs) are typically
used for behavior planning given natural language instructions from the user.
However, dealing with ambiguous instructions in real-world environments remains
a challenge for LLMs. Various methods for task ambiguity detection have been
proposed. However, it is difficult to compare them because they are tested on
different datasets and there is no universal benchmark. For this reason, we
propose AmbiK (Ambiguous Tasks in Kitchen Environment), the fully textual
dataset of ambiguous instructions addressed to a robot in a kitchen
environment. AmbiK was collected with the assistance of LLMs and is
human-validated. It comprises 1000 pairs of ambiguous tasks and their
unambiguous counterparts, categorized by ambiguity type (Human Preferences,
Common Sense Knowledge, Safety), with environment descriptions, clarifying
questions and answers, user intents, and task plans, for a total of 2000 tasks.
We hope that AmbiK will enable researchers to perform a unified comparison of
ambiguity detection methods. AmbiK is available at
https://github.com/cog-model/AmbiK-dataset.
AmbiK, a textual dataset of ambiguous instructions for kitchen robots, enables unified comparison of ambiguity detection methods.
Large Language Models LLMs behavior planning ambiguous instructions task ambiguity detection AmbiK dataset human-validated ambiguity types Human Preferences Common Sense Knowledge Safety environment descriptions clarifying questions user intents task plans