title: REFLECT: Summarizing Robot Experiences for Failure Explanation and
    Correction
summary: The ability to detect and analyze failed executions automatically is crucial
  for an explainable and robust robotic system. Recently, Large Language Models
  (LLMs) have demonstrated strong common sense reasoning skills on textual
  inputs. To leverage the power of LLM for robot failure explanation, we propose
  a framework REFLECT, which converts multi-sensory data into a hierarchical
  summary of robot past experiences and queries LLM with a progressive failure
  explanation algorithm. Conditioned on the explanation, a failure correction
  planner generates an executable plan for the robot to correct the failure and
  complete the task. To systematically evaluate the framework, we create the
  RoboFail dataset and show that our LLM-based framework is able to generate
  informative failure explanations that assist successful correction planning.
  Project website: https://roboreflect.github.io/
paper_authors._id: 649cf7e2bc919c0fd7450003
paper_authors.name: Zeyi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-29T11:21:33.733Z
paper_authors._id: 6369a7c856d1f9349812c2ae
paper_authors.avatarUrl: /avatars/0e949ce509c13084c56755cbe4e501b4.svg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Zeyi
paper_authors._id: 649cf7e2bc919c0fd7450004
paper_authors.name: Arpit Bahety
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-29T20:55:37.683Z
paper_authors._id: 649d94cb9208f27d63257152
paper_authors.avatarUrl: /avatars/5518a1ac8827c7c9b1f736ecbcc5e51d.svg
paper_authors.fullname: Arpit Bahety
paper_authors.type: user
paper_authors.user: abahety
paper_authors._id: 649cf7e2bc919c0fd7450005
paper_authors.name: Shuran Song
paper_title: REFLECT: Summarizing Robot Experiences for Failure Explanation and
    Correction
paper_summary: The ability to detect and analyze failed executions automatically is crucial
  for an explainable and robust robotic system. Recently, Large Language Models
  (LLMs) have demonstrated strong common sense reasoning skills on textual
  inputs. To leverage the power of LLM for robot failure explanation, we propose
  a framework REFLECT, which converts multi-sensory data into a hierarchical
  summary of robot past experiences and queries LLM with a progressive failure
  explanation algorithm. Conditioned on the explanation, a failure correction
  planner generates an executable plan for the robot to correct the failure and
  complete the task. To systematically evaluate the framework, we create the
  RoboFail dataset and show that our LLM-based framework is able to generate
  informative failure explanations that assist successful correction planning.
  Project website: https://roboreflect.github.io/
paper_ai_summary: A framework called REFLECT uses LLMs to generate informative failure explanations from multi-sensory data, aiding in correction planning for robots.
paper_ai_keywords: Large Language Models REFLECT multi-sensory data failure explanation algorithm failure correction planner hierarchical summary RoboFail dataset