Real-World Fluid Directed Rigid Body Control via Deep Reinforcement
  Learning
Recent advances in real-world applications of reinforcement learning (RL)
have relied on the ability to accurately simulate systems at scale. However,
domains such as fluid dynamical systems exhibit complex dynamic phenomena that
are hard to simulate at high integration rates, limiting the direct application
of modern deep RL algorithms to often expensive or safety critical hardware. In
this work, we introduce "Box o Flows", a novel benchtop experimental control
system for systematically evaluating RL algorithms in dynamic real-world
scenarios. We describe the key components of the Box o Flows, and through a
series of experiments demonstrate how state-of-the-art model-free RL algorithms
can synthesize a variety of complex behaviors via simple reward specifications.
Furthermore, we explore the role of offline RL in data-efficient hypothesis
testing by reusing past experiences. We believe that the insights gained from
this preliminary study and the availability of systems like the Box o Flows
support the way forward for developing systematic RL algorithms that can be
generally applied to complex, dynamical systems. Supplementary material and
videos of experiments are available at
https://sites.google.com/view/box-o-flows/home.
65c9a1f6082c6175884a6037
Mohak Bhardwaj
admin_assigned
2024-02-12T17:12:40.805Z
6373bcb5250cf1379bbdafde
/avatars/c0763f396574588e7bdc6bd4ec119d6b.svg
Mohak Bhardwaj
user
mohakb
65c9a1f6082c6175884a6038
Thomas Lampe
admin_assigned
2024-02-12T17:12:52.377Z
631fa7e9124782a19efd20f2
/avatars/456ef70cdb2a6a1f79a078746e96034a.svg
Thomas Lampe
user
tlampe
65c9a1f6082c6175884a6039
Michael Neunert
65c9a1f6082c6175884a603a
Francesco Romano
65c9a1f6082c6175884a603b
Abbas Abdolmaleki
65c9a1f6082c6175884a603c
Arunkumar Byravan
65c9a1f6082c6175884a603d
Markus Wulfmeier
65c9a1f6082c6175884a603e
Martin Riedmiller
65c9a1f6082c6175884a603f
Jonas Buchli
Real-World Fluid Directed Rigid Body Control via Deep Reinforcement
  Learning
Recent advances in real-world applications of reinforcement learning (RL)
have relied on the ability to accurately simulate systems at scale. However,
domains such as fluid dynamical systems exhibit complex dynamic phenomena that
are hard to simulate at high integration rates, limiting the direct application
of modern deep RL algorithms to often expensive or safety critical hardware. In
this work, we introduce "Box o Flows", a novel benchtop experimental control
system for systematically evaluating RL algorithms in dynamic real-world
scenarios. We describe the key components of the Box o Flows, and through a
series of experiments demonstrate how state-of-the-art model-free RL algorithms
can synthesize a variety of complex behaviors via simple reward specifications.
Furthermore, we explore the role of offline RL in data-efficient hypothesis
testing by reusing past experiences. We believe that the insights gained from
this preliminary study and the availability of systems like the Box o Flows
support the way forward for developing systematic RL algorithms that can be
generally applied to complex, dynamical systems. Supplementary material and
videos of experiments are available at
https://sites.google.com/view/box-o-flows/home.
Box o Flows evaluates reinforcement learning algorithms in real-world dynamic scenarios, demonstrating complex behavior synthesis and the role of offline reinforcement learning in efficient hypothesis testing.
reinforcement learning RL offline RL model-free RL Box o Flows dynamic real-world scenarios complex behaviors simple reward specifications hypothesis testing