title: Robots That Ask For Help: Uncertainty Alignment for Large Language Model
    Planners
summary: Large language models (LLMs) exhibit a wide range of promising capabilities
  -- from step-by-step planning to commonsense reasoning -- that may provide
  utility for robots, but remain prone to confidently hallucinated predictions.
  In this work, we present KnowNo, which is a framework for measuring and
  aligning the uncertainty of LLM-based planners such that they know when they
  don't know and ask for help when needed. KnowNo builds on the theory of
  conformal prediction to provide statistical guarantees on task completion while
  minimizing human help in complex multi-step planning settings. Experiments
  across a variety of simulated and real robot setups that involve tasks with
  different modes of ambiguity (e.g., from spatial to numeric uncertainties, from
  human preferences to Winograd schemas) show that KnowNo performs favorably over
  modern baselines (which may involve ensembles or extensive prompt tuning) in
  terms of improving efficiency and autonomy, while providing formal assurances.
  KnowNo can be used with LLMs out of the box without model-finetuning, and
  suggests a promising lightweight approach to modeling uncertainty that can
  complement and scale with the growing capabilities of foundation models.
  Website: https://robot-help.github.io
paper_authors._id: 64a628d185638d0e698ca8f8
paper_authors.name: Allen Z. Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-06T14:03:52.809Z
paper_authors._id: 6392628841491be61b52a4dc
paper_authors.avatarUrl: /avatars/c1e59fbf2252e332a3c86c2bd18afd09.svg
paper_authors.fullname: Allen Z. Ren
paper_authors.type: user
paper_authors.user: allenzren
paper_authors._id: 64a628d185638d0e698ca8f9
paper_authors.name: Anushri Dixit
paper_authors._id: 64a628d185638d0e698ca8fa
paper_authors.name: Alexandra Bodrova
paper_authors._id: 64a628d185638d0e698ca8fb
paper_authors.name: Sumeet Singh
paper_authors._id: 64a628d185638d0e698ca8fc
paper_authors.name: Stephen Tu
paper_authors._id: 64a628d185638d0e698ca8fd
paper_authors.name: Noah Brown
paper_authors._id: 64a628d185638d0e698ca8fe
paper_authors.name: Peng Xu
paper_authors._id: 64a628d185638d0e698ca8ff
paper_authors.name: Leila Takayama
paper_authors._id: 64a628d185638d0e698ca900
paper_authors.name: Fei Xia
paper_authors._id: 64a628d185638d0e698ca901
paper_authors.name: Jake Varley
paper_authors._id: 64a628d185638d0e698ca902
paper_authors.name: Zhenjia Xu
paper_authors._id: 64a628d185638d0e698ca903
paper_authors.name: Dorsa Sadigh
paper_authors._id: 64a628d185638d0e698ca904
paper_authors.name: Andy Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-06T14:11:06.397Z
paper_authors._id: 62d5f9d705461e9459d20b91
paper_authors.avatarUrl: /avatars/120917da2d8587674505163fb960e27d.svg
paper_authors.fullname: Andy Zeng
paper_authors.type: user
paper_authors.user: andyzeng
paper_authors._id: 64a628d185638d0e698ca905
paper_authors.name: Anirudha Majumdar
paper_title: Robots That Ask For Help: Uncertainty Alignment for Large Language Model
    Planners
paper_summary: Large language models (LLMs) exhibit a wide range of promising capabilities
  -- from step-by-step planning to commonsense reasoning -- that may provide
  utility for robots, but remain prone to confidently hallucinated predictions.
  In this work, we present KnowNo, which is a framework for measuring and
  aligning the uncertainty of LLM-based planners such that they know when they
  don't know and ask for help when needed. KnowNo builds on the theory of
  conformal prediction to provide statistical guarantees on task completion while
  minimizing human help in complex multi-step planning settings. Experiments
  across a variety of simulated and real robot setups that involve tasks with
  different modes of ambiguity (e.g., from spatial to numeric uncertainties, from
  human preferences to Winograd schemas) show that KnowNo performs favorably over
  modern baselines (which may involve ensembles or extensive prompt tuning) in
  terms of improving efficiency and autonomy, while providing formal assurances.
  KnowNo can be used with LLMs out of the box without model-finetuning, and
  suggests a promising lightweight approach to modeling uncertainty that can
  complement and scale with the growing capabilities of foundation models.
  Website: https://robot-help.github.io
paper_ai_keywords: KnowNo conformal prediction LLM-based planners uncertainty statistical guarantees ensemble prompt tuning spatial uncertainties numeric uncertainties human preferences Winograd schemas