title: RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual
    Dexterous Robot Hands
summary: It has been a long-standing research goal to endow robot hands with
  human-level dexterity. Bi-manual robot piano playing constitutes a task that
  combines challenges from dynamic tasks, such as generating fast while precise
  motions, with slower but contact-rich manipulation problems. Although
  reinforcement learning based approaches have shown promising results in
  single-task performance, these methods struggle in a multi-song setting. Our
  work aims to close this gap and, thereby, enable imitation learning approaches
  for robot piano playing at scale. To this end, we introduce the Robot Piano 1
  Million (RP1M) dataset, containing bi-manual robot piano playing motion data of
  more than one million trajectories. We formulate finger placements as an
  optimal transport problem, thus, enabling automatic annotation of vast amounts
  of unlabeled songs. Benchmarking existing imitation learning approaches shows
  that such approaches reach state-of-the-art robot piano playing performance by
  leveraging RP1M.
paper_authors._id: 66c5593fb666ae525586a3ea
paper_authors.name: Yi Zhao
paper_authors._id: 66c5593fb666ae525586a3eb
paper_authors.name: Le Chen
paper_authors._id: 66c5593fb666ae525586a3ec
paper_authors.name: Jan Schneider
paper_authors._id: 66c5593fb666ae525586a3ed
paper_authors.name: Quankai Gao
paper_authors._id: 66c5593fb666ae525586a3ee
paper_authors.name: Juho Kannala
paper_authors._id: 66c5593fb666ae525586a3ef
paper_authors.name: Bernhard Schölkopf
paper_authors._id: 66c5593fb666ae525586a3f0
paper_authors.name: Joni Pajarinen
paper_authors._id: 66c5593fb666ae525586a3f1
paper_authors.name: Dieter Büchler
paper_title: RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual
    Dexterous Robot Hands
paper_summary: It has been a long-standing research goal to endow robot hands with
  human-level dexterity. Bi-manual robot piano playing constitutes a task that
  combines challenges from dynamic tasks, such as generating fast while precise
  motions, with slower but contact-rich manipulation problems. Although
  reinforcement learning based approaches have shown promising results in
  single-task performance, these methods struggle in a multi-song setting. Our
  work aims to close this gap and, thereby, enable imitation learning approaches
  for robot piano playing at scale. To this end, we introduce the Robot Piano 1
  Million (RP1M) dataset, containing bi-manual robot piano playing motion data of
  more than one million trajectories. We formulate finger placements as an
  optimal transport problem, thus, enabling automatic annotation of vast amounts
  of unlabeled songs. Benchmarking existing imitation learning approaches shows
  that such approaches reach state-of-the-art robot piano playing performance by
  leveraging RP1M.
paper_ai_summary: A dataset of over one million bi-manual robot piano playing trajectories enables state-of-the-art imitation learning performance for robot piano playing.
paper_ai_keywords: bi-manual robot piano playing reinforcement learning imitation learning RP1M dataset finger placements optimal transport problem