title: SERL: A Software Suite for Sample-Efficient Robotic Reinforcement
    Learning
summary: In recent years, significant progress has been made in the field of robotic
  reinforcement learning (RL), enabling methods that handle complex image
  observations, train in the real world, and incorporate auxiliary data, such as
  demonstrations and prior experience. However, despite these advances, robotic
  RL remains hard to use. It is acknowledged among practitioners that the
  particular implementation details of these algorithms are often just as
  important (if not more so) for performance as the choice of algorithm. We posit
  that a significant challenge to widespread adoption of robotic RL, as well as
  further development of robotic RL methods, is the comparative inaccessibility
  of such methods. To address this challenge, we developed a carefully
  implemented library containing a sample efficient off-policy deep RL method,
  together with methods for computing rewards and resetting the environment, a
  high-quality controller for a widely-adopted robot, and a number of challenging
  example tasks. We provide this library as a resource for the community,
  describe its design choices, and present experimental results. Perhaps
  surprisingly, we find that our implementation can achieve very efficient
  learning, acquiring policies for PCB board assembly, cable routing, and object
  relocation between 25 to 50 minutes of training per policy on average,
  improving over state-of-the-art results reported for similar tasks in the
  literature. These policies achieve perfect or near-perfect success rates,
  extreme robustness even under perturbations, and exhibit emergent recovery and
  correction behaviors. We hope that these promising results and our high-quality
  open-source implementation will provide a tool for the robotics community to
  facilitate further developments in robotic RL. Our code, documentation, and
  videos can be found at https://serl-robot.github.io/
paper_authors._id: 65b88b1954cfdff8f8814022
paper_authors.name: Jianlan Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-30T08:06:13.647Z
paper_authors._id: 64f8cb8ed04a890f5380d9a4
paper_authors.avatarUrl: /avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
paper_authors.fullname: Jianlan Luo
paper_authors.type: user
paper_authors.user: jianlanluo
paper_authors._id: 65b88b1954cfdff8f8814023
paper_authors.name: Zheyuan Hu
paper_authors._id: 65b88b1954cfdff8f8814024
paper_authors.name: Charles Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:46:26.033Z
paper_authors._id: 658fdaacac02633c0d78e3e8
paper_authors.avatarUrl: /avatars/c63481494f86368185a9e33587de77d3.svg
paper_authors.fullname: Charles Xu
paper_authors.type: user
paper_authors.user: charlesxu0124
paper_authors._id: 65b88b1954cfdff8f8814025
paper_authors.name: You Liang Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:46:32.072Z
paper_authors._id: 635dcbee79599a4f7db46db3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635dcbee79599a4f7db46db3/eIi5cPQelHi0TkNNe1_XT.jpeg
paper_authors.fullname: You Liang Tan
paper_authors.type: user
paper_authors.user: youliangtan
paper_authors._id: 65b88b1954cfdff8f8814026
paper_authors.name: Jacob Berg
paper_authors._id: 65b88b1954cfdff8f8814027
paper_authors.name: Archit Sharma
paper_authors._id: 65b88b1954cfdff8f8814028
paper_authors.name: Stefan Schaal
paper_authors._id: 65b88b1954cfdff8f8814029
paper_authors.name: Chelsea Finn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:47:29.290Z
paper_authors._id: 64ac22a9193f0a807deb673d
paper_authors.avatarUrl: /avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
paper_authors.fullname: Chelsea Finn
paper_authors.type: user
paper_authors.user: cbfinn
paper_authors._id: 65b88b1954cfdff8f881402a
paper_authors.name: Abhishek Gupta
paper_authors._id: 65b88b1954cfdff8f881402b
paper_authors.name: Sergey Levine
paper_title: SERL: A Software Suite for Sample-Efficient Robotic Reinforcement
    Learning
paper_summary: In recent years, significant progress has been made in the field of robotic
  reinforcement learning (RL), enabling methods that handle complex image
  observations, train in the real world, and incorporate auxiliary data, such as
  demonstrations and prior experience. However, despite these advances, robotic
  RL remains hard to use. It is acknowledged among practitioners that the
  particular implementation details of these algorithms are often just as
  important (if not more so) for performance as the choice of algorithm. We posit
  that a significant challenge to widespread adoption of robotic RL, as well as
  further development of robotic RL methods, is the comparative inaccessibility
  of such methods. To address this challenge, we developed a carefully
  implemented library containing a sample efficient off-policy deep RL method,
  together with methods for computing rewards and resetting the environment, a
  high-quality controller for a widely-adopted robot, and a number of challenging
  example tasks. We provide this library as a resource for the community,
  describe its design choices, and present experimental results. Perhaps
  surprisingly, we find that our implementation can achieve very efficient
  learning, acquiring policies for PCB board assembly, cable routing, and object
  relocation between 25 to 50 minutes of training per policy on average,
  improving over state-of-the-art results reported for similar tasks in the
  literature. These policies achieve perfect or near-perfect success rates,
  extreme robustness even under perturbations, and exhibit emergent recovery and
  correction behaviors. We hope that these promising results and our high-quality
  open-source implementation will provide a tool for the robotics community to
  facilitate further developments in robotic RL. Our code, documentation, and
  videos can be found at https://serl-robot.github.io/
paper_ai_summary: A library for robotic reinforcement learning provides efficient off-policy deep RL methods, high-quality controllers, and example tasks, achieving state-of-the-art results in tasks like PCB board assembly and cable routing.
paper_ai_keywords: off-policy deep RL rewards environment reset high-quality controller emergent recovery