title: AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
summary: As a part of an embodied agent, Large Language Models (LLMs) are typically
  used for behavior planning given natural language instructions from the user.
  However, dealing with ambiguous instructions in real-world environments remains
  a challenge for LLMs. Various methods for task ambiguity detection have been
  proposed. However, it is difficult to compare them because they are tested on
  different datasets and there is no universal benchmark. For this reason, we
  propose AmbiK (Ambiguous Tasks in Kitchen Environment), the fully textual
  dataset of ambiguous instructions addressed to a robot in a kitchen
  environment. AmbiK was collected with the assistance of LLMs and is
  human-validated. It comprises 1000 pairs of ambiguous tasks and their
  unambiguous counterparts, categorized by ambiguity type (Human Preferences,
  Common Sense Knowledge, Safety), with environment descriptions, clarifying
  questions and answers, user intents, and task plans, for a total of 2000 tasks.
  We hope that AmbiK will enable researchers to perform a unified comparison of
  ambiguity detection methods. AmbiK is available at
  https://github.com/cog-model/AmbiK-dataset.
paper_authors._id: 684153cf911d1b3135fa5dfe
paper_authors.name: Anastasiia Ivanova
paper_authors._id: 684153cf911d1b3135fa5dff
paper_authors.name: Eva Bakaeva
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T10:00:02.435Z
paper_authors._id: 661af24d8328f43c6abc2d11
paper_authors.avatarUrl: /avatars/afe7eaf1f7a378dbcdba5cd3e86adf9c.svg
paper_authors.fullname: Eva
paper_authors.type: user
paper_authors.user: tenebrissilvam
paper_authors._id: 684153cf911d1b3135fa5e00
paper_authors.name: Zoya Volovikova
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-06-05T08:22:39.926Z
paper_authors._id: 64198f70ed725fef6442b37e
paper_authors.avatarUrl: /avatars/580ab07a3067a9deb2977b0894226fe3.svg
paper_authors.fullname: Alexey Kovalev
paper_authors.type: user
paper_authors.user: AlexeyKov
paper_authors._id: 684153cf911d1b3135fa5e01
paper_authors.name: Alexey K. Kovalev
paper_authors._id: 684153cf911d1b3135fa5e02
paper_authors.name: Aleksandr I. Panov
paper_title: AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
paper_summary: As a part of an embodied agent, Large Language Models (LLMs) are typically
  used for behavior planning given natural language instructions from the user.
  However, dealing with ambiguous instructions in real-world environments remains
  a challenge for LLMs. Various methods for task ambiguity detection have been
  proposed. However, it is difficult to compare them because they are tested on
  different datasets and there is no universal benchmark. For this reason, we
  propose AmbiK (Ambiguous Tasks in Kitchen Environment), the fully textual
  dataset of ambiguous instructions addressed to a robot in a kitchen
  environment. AmbiK was collected with the assistance of LLMs and is
  human-validated. It comprises 1000 pairs of ambiguous tasks and their
  unambiguous counterparts, categorized by ambiguity type (Human Preferences,
  Common Sense Knowledge, Safety), with environment descriptions, clarifying
  questions and answers, user intents, and task plans, for a total of 2000 tasks.
  We hope that AmbiK will enable researchers to perform a unified comparison of
  ambiguity detection methods. AmbiK is available at
  https://github.com/cog-model/AmbiK-dataset.
paper_ai_summary: AmbiK, a textual dataset of ambiguous instructions for kitchen robots, enables unified comparison of ambiguity detection methods.
paper_ai_keywords: Large Language Models LLMs behavior planning ambiguous instructions task ambiguity detection AmbiK dataset human-validated ambiguity types Human Preferences Common Sense Knowledge Safety environment descriptions clarifying questions user intents task plans