title: Real-World Fluid Directed Rigid Body Control via Deep Reinforcement
    Learning
summary: Recent advances in real-world applications of reinforcement learning (RL)
  have relied on the ability to accurately simulate systems at scale. However,
  domains such as fluid dynamical systems exhibit complex dynamic phenomena that
  are hard to simulate at high integration rates, limiting the direct application
  of modern deep RL algorithms to often expensive or safety critical hardware. In
  this work, we introduce "Box o Flows", a novel benchtop experimental control
  system for systematically evaluating RL algorithms in dynamic real-world
  scenarios. We describe the key components of the Box o Flows, and through a
  series of experiments demonstrate how state-of-the-art model-free RL algorithms
  can synthesize a variety of complex behaviors via simple reward specifications.
  Furthermore, we explore the role of offline RL in data-efficient hypothesis
  testing by reusing past experiences. We believe that the insights gained from
  this preliminary study and the availability of systems like the Box o Flows
  support the way forward for developing systematic RL algorithms that can be
  generally applied to complex, dynamical systems. Supplementary material and
  videos of experiments are available at
  https://sites.google.com/view/box-o-flows/home.
paper_authors._id: 65c9a1f6082c6175884a6037
paper_authors.name: Mohak Bhardwaj
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T17:12:40.805Z
paper_authors._id: 6373bcb5250cf1379bbdafde
paper_authors.avatarUrl: /avatars/c0763f396574588e7bdc6bd4ec119d6b.svg
paper_authors.fullname: Mohak Bhardwaj
paper_authors.type: user
paper_authors.user: mohakb
paper_authors._id: 65c9a1f6082c6175884a6038
paper_authors.name: Thomas Lampe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T17:12:52.377Z
paper_authors._id: 631fa7e9124782a19efd20f2
paper_authors.avatarUrl: /avatars/456ef70cdb2a6a1f79a078746e96034a.svg
paper_authors.fullname: Thomas Lampe
paper_authors.type: user
paper_authors.user: tlampe
paper_authors._id: 65c9a1f6082c6175884a6039
paper_authors.name: Michael Neunert
paper_authors._id: 65c9a1f6082c6175884a603a
paper_authors.name: Francesco Romano
paper_authors._id: 65c9a1f6082c6175884a603b
paper_authors.name: Abbas Abdolmaleki
paper_authors._id: 65c9a1f6082c6175884a603c
paper_authors.name: Arunkumar Byravan
paper_authors._id: 65c9a1f6082c6175884a603d
paper_authors.name: Markus Wulfmeier
paper_authors._id: 65c9a1f6082c6175884a603e
paper_authors.name: Martin Riedmiller
paper_authors._id: 65c9a1f6082c6175884a603f
paper_authors.name: Jonas Buchli
paper_title: Real-World Fluid Directed Rigid Body Control via Deep Reinforcement
    Learning
paper_summary: Recent advances in real-world applications of reinforcement learning (RL)
  have relied on the ability to accurately simulate systems at scale. However,
  domains such as fluid dynamical systems exhibit complex dynamic phenomena that
  are hard to simulate at high integration rates, limiting the direct application
  of modern deep RL algorithms to often expensive or safety critical hardware. In
  this work, we introduce "Box o Flows", a novel benchtop experimental control
  system for systematically evaluating RL algorithms in dynamic real-world
  scenarios. We describe the key components of the Box o Flows, and through a
  series of experiments demonstrate how state-of-the-art model-free RL algorithms
  can synthesize a variety of complex behaviors via simple reward specifications.
  Furthermore, we explore the role of offline RL in data-efficient hypothesis
  testing by reusing past experiences. We believe that the insights gained from
  this preliminary study and the availability of systems like the Box o Flows
  support the way forward for developing systematic RL algorithms that can be
  generally applied to complex, dynamical systems. Supplementary material and
  videos of experiments are available at
  https://sites.google.com/view/box-o-flows/home.
paper_ai_summary: Box o Flows evaluates reinforcement learning algorithms in real-world dynamic scenarios, demonstrating complex behavior synthesis and the role of offline reinforcement learning in efficient hypothesis testing.
paper_ai_keywords: reinforcement learning RL offline RL model-free RL Box o Flows dynamic real-world scenarios complex behaviors simple reward specifications hypothesis testing