Vision-Language Models as a Source of Rewards
Building generalist agents that can accomplish many goals in rich open-ended
environments is one of the research frontiers for reinforcement learning. A key
limiting factor for building generalist agents with RL has been the need for a
large number of reward functions for achieving different goals. We investigate
the feasibility of using off-the-shelf vision-language models, or VLMs, as
sources of rewards for reinforcement learning agents. We show how rewards for
visual achievement of a variety of language goals can be derived from the CLIP
family of models, and used to train RL agents that can achieve a variety of
language goals. We showcase this approach in two distinct visual domains and
present a scaling trend showing how larger VLMs lead to more accurate rewards
for visual goal achievement, which in turn produces more capable RL agents.
657bcd1d7e11375c6f27ed4e
Kate Baumli
657bcd1d7e11375c6f27ed4f
Satinder Baveja
657bcd1d7e11375c6f27ed50
Feryal Behbahani
extracted_confirmed
2024-09-20T07:25:16.632Z
64c197b7ec3c6181351b2c5d
/avatars/f81fe98f68720b95885d4968b0e7cc0f.svg
Feryal Behbahani
user
ferya
657bcd1d7e11375c6f27ed51
Harris Chan
admin_assigned
2023-12-15T10:56:04.792Z
62d1cf9339e0332b575180ef
/avatars/e43acba1e9d21564fb82835a189624ab.svg
Harris Chan
user
takonan
657bcd1d7e11375c6f27ed52
Gheorghe Comanici
657bcd1d7e11375c6f27ed53
Sebastian Flennerhag
657bcd1d7e11375c6f27ed54
Maxime Gazeau
657bcd1d7e11375c6f27ed55
Kristian Holsheimer
657bcd1d7e11375c6f27ed56
Dan Horgan
657bcd1d7e11375c6f27ed57
Michael Laskin
657bcd1d7e11375c6f27ed58
Clare Lyle
admin_assigned
2023-12-15T10:57:07.252Z
6313f69d54e6e5d9f0fb9f10
/avatars/12114d479bb6dc394d29f988944f6d47.svg
Clare Lyle
user
justclarifying
657bcd1d7e11375c6f27ed59
Hussain Masoom
657bcd1d7e11375c6f27ed5a
Kay McKinney
657bcd1d7e11375c6f27ed5b
Volodymyr Mnih
657bcd1d7e11375c6f27ed5c
Alexander Neitz
657bcd1d7e11375c6f27ed5d
Fabio Pardo
657bcd1d7e11375c6f27ed5e
Jack Parker-Holder
657bcd1d7e11375c6f27ed5f
John Quan
657bcd1d7e11375c6f27ed60
Tim Rocktäschel
657bcd1d7e11375c6f27ed61
Himanshu Sahni
657bcd1d7e11375c6f27ed62
Tom Schaul
657bcd1d7e11375c6f27ed63
Yannick Schroecker
657bcd1d7e11375c6f27ed64
Stephen Spencer
657bcd1d7e11375c6f27ed65
Richie Steigerwald
657bcd1d7e11375c6f27ed66
Luyu Wang
657bcd1d7e11375c6f27ed67
Lei Zhang
Vision-Language Models as a Source of Rewards
Building generalist agents that can accomplish many goals in rich open-ended
environments is one of the research frontiers for reinforcement learning. A key
limiting factor for building generalist agents with RL has been the need for a
large number of reward functions for achieving different goals. We investigate
the feasibility of using off-the-shelf vision-language models, or VLMs, as
sources of rewards for reinforcement learning agents. We show how rewards for
visual achievement of a variety of language goals can be derived from the CLIP
family of models, and used to train RL agents that can achieve a variety of
language goals. We showcase this approach in two distinct visual domains and
present a scaling trend showing how larger VLMs lead to more accurate rewards
for visual goal achievement, which in turn produces more capable RL agents.
Researchers explore using off-the-shelf vision-language models to derive rewards for reinforcement learning agents, demonstrating improved performance across various language-based visual goals.
reinforcement learning generalist agents reward functions vision-language models VLMs CLIP visual achievement language goals scaling trend capable RL agents