title: Decomposing the Generalization Gap in Imitation Learning for Visual
    Robotic Manipulation
summary: What makes generalization hard for imitation learning in visual robotic
  manipulation? This question is difficult to approach at face value, but the
  environment from the perspective of a robot can often be decomposed into
  enumerable factors of variation, such as the lighting conditions or the
  placement of the camera. Empirically, generalization to some of these factors
  have presented a greater obstacle than others, but existing work sheds little
  light on precisely how much each factor contributes to the generalization gap.
  Towards an answer to this question, we study imitation learning policies in
  simulation and on a real robot language-conditioned manipulation task to
  quantify the difficulty of generalization to different (sets of) factors. We
  also design a new simulated benchmark of 19 tasks with 11 factors of variation
  to facilitate more controlled evaluations of generalization. From our study, we
  determine an ordering of factors based on generalization difficulty, that is
  consistent across simulation and our real robot setup.
paper_authors._id: 64ab998b31f620201d04bcfc
paper_authors.name: Annie Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-10T09:59:35.854Z
paper_authors._id: 649e23bb3e5e75047640b3b5
paper_authors.avatarUrl: /avatars/019d008a5bd7057b8665e1ef856f13a7.svg
paper_authors.fullname: Annie Xie
paper_authors.type: user
paper_authors.user: anniexie
paper_authors._id: 64ab998b31f620201d04bcfd
paper_authors.name: Lisa Lee
paper_authors._id: 64ab998b31f620201d04bcfe
paper_authors.name: Ted Xiao
paper_authors._id: 64ab998b31f620201d04bcff
paper_authors.name: Chelsea Finn
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-11T07:54:07.806Z
paper_authors._id: 64ac22a9193f0a807deb673d
paper_authors.avatarUrl: /avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
paper_authors.fullname: Chelsea Finn
paper_authors.type: user
paper_authors.user: cbfinn
paper_title: Decomposing the Generalization Gap in Imitation Learning for Visual
    Robotic Manipulation
paper_summary: What makes generalization hard for imitation learning in visual robotic
  manipulation? This question is difficult to approach at face value, but the
  environment from the perspective of a robot can often be decomposed into
  enumerable factors of variation, such as the lighting conditions or the
  placement of the camera. Empirically, generalization to some of these factors
  have presented a greater obstacle than others, but existing work sheds little
  light on precisely how much each factor contributes to the generalization gap.
  Towards an answer to this question, we study imitation learning policies in
  simulation and on a real robot language-conditioned manipulation task to
  quantify the difficulty of generalization to different (sets of) factors. We
  also design a new simulated benchmark of 19 tasks with 11 factors of variation
  to facilitate more controlled evaluations of generalization. From our study, we
  determine an ordering of factors based on generalization difficulty, that is
  consistent across simulation and our real robot setup.
paper_ai_summary: The study investigates the generalization challenges in imitation learning for visual robotic manipulation by quantifying the impact of different factors of variation in both simulation and real-world settings.
paper_ai_keywords: imitation learning visual robotic manipulation factors of variation generalization difficulty