Learning to Identify Critical States for Reinforcement Learning from
  Videos
Recent work on deep reinforcement learning (DRL) has pointed out that
algorithmic information about good policies can be extracted from offline data
which lack explicit information about executed actions. For example, videos of
humans or robots may convey a lot of implicit information about rewarding
action sequences, but a DRL machine that wants to profit from watching such
videos must first learn by itself to identify and recognize relevant
states/actions/rewards. Without relying on ground-truth annotations, our new
method called Deep State Identifier learns to predict returns from episodes
encoded as videos. Then it uses a kind of mask-based sensitivity analysis to
extract/identify important critical states. Extensive experiments showcase our
method's potential for understanding and improving agent behavior. The source
code and the generated datasets are available at
https://github.com/AI-Initiative-KAUST/VideoRLCS.
64dca559ea8c6869315c3443
Haozhe Liu
64dca559ea8c6869315c3444
Mingchen Zhuge
admin_assigned
2023-08-20T14:05:51.821Z
64403daae44f30a72323e4ca
https://cdn-avatars.huggingface.co/v1/production/uploads/64403daae44f30a72323e4ca/skJ9h0pdNfmE4VbQL8xDR.png
mingchen zhuge
user
tjpxiaoming
64dca559ea8c6869315c3445
Bing Li
claimed_verified
2025-03-31T08:17:50.311Z
666ddb45c0f3d5afc27e85ba
/avatars/dce98fc77bd8cf9e348f2d91bc3c0225.svg
Bing Li
user
bing-li-ai
64dca559ea8c6869315c3446
Yuhui Wang
64dca559ea8c6869315c3447
Francesco Faccio
64dca559ea8c6869315c3448
Bernard Ghanem
64dca559ea8c6869315c3449
Jürgen Schmidhuber
Learning to Identify Critical States for Reinforcement Learning from
  Videos
Recent work on deep reinforcement learning (DRL) has pointed out that
algorithmic information about good policies can be extracted from offline data
which lack explicit information about executed actions. For example, videos of
humans or robots may convey a lot of implicit information about rewarding
action sequences, but a DRL machine that wants to profit from watching such
videos must first learn by itself to identify and recognize relevant
states/actions/rewards. Without relying on ground-truth annotations, our new
method called Deep State Identifier learns to predict returns from episodes
encoded as videos. Then it uses a kind of mask-based sensitivity analysis to
extract/identify important critical states. Extensive experiments showcase our
method's potential for understanding and improving agent behavior. The source
code and the generated datasets are available at
https://github.com/AI-Initiative-KAUST/VideoRLCS.
A new method in deep reinforcement learning extracts important states from unannotated video data to improve policy learning and agent behavior.
deep reinforcement learning DRL offline data good policies videos implicit information rewarding action sequences returns episodes mask-based sensitivity analysis critical states