title: TidyBot: Personalized Robot Assistance with Large Language Models
summary: For a robot to personalize physical assistance effectively, it must learn
  user preferences that can be generally reapplied to future scenarios. In this
  work, we investigate personalization of household cleanup with robots that can
  tidy up rooms by picking up objects and putting them away. A key challenge is
  determining the proper place to put each object, as people's preferences can
  vary greatly depending on personal taste or cultural background. For instance,
  one person may prefer storing shirts in the drawer, while another may prefer
  them on the shelf. We aim to build systems that can learn such preferences from
  just a handful of examples via prior interactions with a particular person. We
  show that robots can combine language-based planning and perception with the
  few-shot summarization capabilities of large language models (LLMs) to infer
  generalized user preferences that are broadly applicable to future
  interactions. This approach enables fast adaptation and achieves 91.2% accuracy
  on unseen objects in our benchmark dataset. We also demonstrate our approach on
  a real-world mobile manipulator called TidyBot, which successfully puts away
  85.0% of objects in real-world test scenarios.
paper_authors._id: 645afad0333fb183578705a0
paper_authors.name: Jimmy Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T07:03:03.205Z
paper_authors._id: 62faae5686456e533453422c
paper_authors.avatarUrl: /avatars/78e175e131fe472257c2f93787e463a1.svg
paper_authors.fullname: Jimmy Wu
paper_authors.type: user
paper_authors.user: jimmyyhwu
paper_authors._id: 645afad0333fb183578705a1
paper_authors.name: Rika Antonova
paper_authors._id: 645afad0333fb183578705a2
paper_authors.name: Adam Kan
paper_authors._id: 645afad0333fb183578705a3
paper_authors.name: Marion Lepert
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-05-10T02:01:35.028Z
paper_authors._id: 63d9b55faefaa735258522f5
paper_authors.avatarUrl: /avatars/5b5eb3dc922484db2d2646f0a435b338.svg
paper_authors.fullname: Marion Lepert
paper_authors.type: user
paper_authors.user: lepertm
paper_authors._id: 645afad0333fb183578705a4
paper_authors.name: Andy Zeng
paper_authors._id: 645afad0333fb183578705a5
paper_authors.name: Shuran Song
paper_authors._id: 645afad0333fb183578705a6
paper_authors.name: Jeannette Bohg
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-18T09:55:47.853Z
paper_authors._id: 676272a59865895e253ceee5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/676272a59865895e253ceee5/Nz51Y-G49SD19lQEjH6Yd.jpeg
paper_authors.fullname: Jeannette Bohg
paper_authors.type: user
paper_authors.user: jbohg
paper_authors._id: 645afad0333fb183578705a7
paper_authors.name: Szymon Rusinkiewicz
paper_authors._id: 645afad0333fb183578705a8
paper_authors.name: Thomas Funkhouser
paper_title: TidyBot: Personalized Robot Assistance with Large Language Models
paper_summary: For a robot to personalize physical assistance effectively, it must learn
  user preferences that can be generally reapplied to future scenarios. In this
  work, we investigate personalization of household cleanup with robots that can
  tidy up rooms by picking up objects and putting them away. A key challenge is
  determining the proper place to put each object, as people's preferences can
  vary greatly depending on personal taste or cultural background. For instance,
  one person may prefer storing shirts in the drawer, while another may prefer
  them on the shelf. We aim to build systems that can learn such preferences from
  just a handful of examples via prior interactions with a particular person. We
  show that robots can combine language-based planning and perception with the
  few-shot summarization capabilities of large language models (LLMs) to infer
  generalized user preferences that are broadly applicable to future
  interactions. This approach enables fast adaptation and achieves 91.2% accuracy
  on unseen objects in our benchmark dataset. We also demonstrate our approach on
  a real-world mobile manipulator called TidyBot, which successfully puts away
  85.0% of objects in real-world test scenarios.
paper_ai_keywords: language-based planning perception few-shot summarization large language models (LLMs)