title: Learning to Identify Critical States for Reinforcement Learning from
    Videos
summary: Recent work on deep reinforcement learning (DRL) has pointed out that
  algorithmic information about good policies can be extracted from offline data
  which lack explicit information about executed actions. For example, videos of
  humans or robots may convey a lot of implicit information about rewarding
  action sequences, but a DRL machine that wants to profit from watching such
  videos must first learn by itself to identify and recognize relevant
  states/actions/rewards. Without relying on ground-truth annotations, our new
  method called Deep State Identifier learns to predict returns from episodes
  encoded as videos. Then it uses a kind of mask-based sensitivity analysis to
  extract/identify important critical states. Extensive experiments showcase our
  method's potential for understanding and improving agent behavior. The source
  code and the generated datasets are available at
  https://github.com/AI-Initiative-KAUST/VideoRLCS.
paper_authors._id: 64dca559ea8c6869315c3443
paper_authors.name: Haozhe Liu
paper_authors._id: 64dca559ea8c6869315c3444
paper_authors.name: Mingchen Zhuge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-20T14:05:51.821Z
paper_authors._id: 64403daae44f30a72323e4ca
paper_authors.avatarUrl: /avatars/c7dd20b594595cdfd56f709dce68b474.svg
paper_authors.fullname: mingchen zhuge
paper_authors.type: user
paper_authors.user: tjpxiaoming
paper_authors._id: 64dca559ea8c6869315c3445
paper_authors.name: Bing Li
paper_authors._id: 64dca559ea8c6869315c3446
paper_authors.name: Yuhui Wang
paper_authors._id: 64dca559ea8c6869315c3447
paper_authors.name: Francesco Faccio
paper_authors._id: 64dca559ea8c6869315c3448
paper_authors.name: Bernard Ghanem
paper_authors._id: 64dca559ea8c6869315c3449
paper_authors.name: Jürgen Schmidhuber
paper_title: Learning to Identify Critical States for Reinforcement Learning from
    Videos
paper_summary: Recent work on deep reinforcement learning (DRL) has pointed out that
  algorithmic information about good policies can be extracted from offline data
  which lack explicit information about executed actions. For example, videos of
  humans or robots may convey a lot of implicit information about rewarding
  action sequences, but a DRL machine that wants to profit from watching such
  videos must first learn by itself to identify and recognize relevant
  states/actions/rewards. Without relying on ground-truth annotations, our new
  method called Deep State Identifier learns to predict returns from episodes
  encoded as videos. Then it uses a kind of mask-based sensitivity analysis to
  extract/identify important critical states. Extensive experiments showcase our
  method's potential for understanding and improving agent behavior. The source
  code and the generated datasets are available at
  https://github.com/AI-Initiative-KAUST/VideoRLCS.
paper_ai_keywords: deep reinforcement learning (DRL) good policies offline data implicit information action sequences Deep State Identifier returns episodes videos mask-based sensitivity analysis critical states agent behavior