title: BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human
    Racing Gameplay
summary: Imitation learning learns a policy from demonstrations without requiring
  hand-designed reward functions. In many robotic tasks, such as autonomous
  racing, imitated policies must model complex environment dynamics and human
  decision-making. Sequence modeling is highly effective in capturing intricate
  patterns of motion sequences but struggles to adapt to new environments or
  distribution shifts that are common in real-world robotics tasks. In contrast,
  Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles
  with sample inefficiency and handling complex motion patterns. Thus, we propose
  BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a
  Behavior Transformer (BeT) policy from human demonstrations with online AIL.
  BeTAIL adds an AIL residual policy to the BeT policy to model the sequential
  decision-making process of human experts and correct for out-of-distribution
  states or shifts in environment dynamics. We test BeTAIL on three challenges
  with expert-level demonstrations of real human gameplay in Gran Turismo Sport.
  Our proposed residual BeTAIL reduces environment interactions and improves
  racing performance and stability, even when the BeT is pretrained on different
  tracks than downstream learning. Videos and code available at:
  https://sites.google.com/berkeley.edu/BeTAIL/home.
paper_authors._id: 65d8224b984cc240f21e0743
paper_authors.name: Catherine Weaver
paper_authors._id: 65d8224b984cc240f21e0744
paper_authors.name: Chen Tang
paper_authors._id: 65d8224b984cc240f21e0745
paper_authors.name: Ce Hao
paper_authors._id: 65d8224b984cc240f21e0746
paper_authors.name: Kenta Kawamoto
paper_authors._id: 65d8224b984cc240f21e0747
paper_authors.name: Masayoshi Tomizuka
paper_authors._id: 65d8224b984cc240f21e0748
paper_authors.name: Wei Zhan
paper_title: BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human
    Racing Gameplay
paper_summary: Imitation learning learns a policy from demonstrations without requiring
  hand-designed reward functions. In many robotic tasks, such as autonomous
  racing, imitated policies must model complex environment dynamics and human
  decision-making. Sequence modeling is highly effective in capturing intricate
  patterns of motion sequences but struggles to adapt to new environments or
  distribution shifts that are common in real-world robotics tasks. In contrast,
  Adversarial Imitation Learning (AIL) can mitigate this effect, but struggles
  with sample inefficiency and handling complex motion patterns. Thus, we propose
  BeTAIL: Behavior Transformer Adversarial Imitation Learning, which combines a
  Behavior Transformer (BeT) policy from human demonstrations with online AIL.
  BeTAIL adds an AIL residual policy to the BeT policy to model the sequential
  decision-making process of human experts and correct for out-of-distribution
  states or shifts in environment dynamics. We test BeTAIL on three challenges
  with expert-level demonstrations of real human gameplay in Gran Turismo Sport.
  Our proposed residual BeTAIL reduces environment interactions and improves
  racing performance and stability, even when the BeT is pretrained on different
  tracks than downstream learning. Videos and code available at:
  https://sites.google.com/berkeley.edu/BeTAIL/home.
paper_ai_summary: BeTAIL combines Behavior Transformer with Adversarial Imitation Learning to enhance robotic imitation in dynamic environments, improving performance and stability with sample efficiency.
paper_ai_keywords: imitation learning Adversarial Imitation Learning (AIL) Behavior Transformer (BeT) residual policy sequential decision-making human experts environment dynamics distribution shifts real-world robotics tasks expert-level demonstrations Gran Turismo Sport