title: Eventful Transformers: Leveraging Temporal Redundancy in Vision
    Transformers
summary: Vision Transformers achieve impressive accuracy across a range of visual
  recognition tasks. Unfortunately, their accuracy frequently comes with high
  computational costs. This is a particular issue in video recognition, where
  models are often applied repeatedly across frames or temporal chunks. In this
  work, we exploit temporal redundancy between subsequent inputs to reduce the
  cost of Transformers for video processing. We describe a method for identifying
  and re-processing only those tokens that have changed significantly over time.
  Our proposed family of models, Eventful Transformers, can be converted from
  existing Transformers (often without any re-training) and give adaptive control
  over the compute cost at runtime. We evaluate our method on large-scale
  datasets for video object detection (ImageNet VID) and action recognition
  (EPIC-Kitchens 100). Our approach leads to significant computational savings
  (on the order of 2-4x) with only minor reductions in accuracy.
paper_authors._id: 64ec7b155ff6f83378095381
paper_authors.name: Matthew Dutson
paper_authors._id: 64ec7b155ff6f83378095382
paper_authors.name: Yin Li
paper_authors._id: 64ec7b155ff6f83378095383
paper_authors.name: Mohit Gupta
paper_title: Eventful Transformers: Leveraging Temporal Redundancy in Vision
    Transformers
paper_summary: Vision Transformers achieve impressive accuracy across a range of visual
  recognition tasks. Unfortunately, their accuracy frequently comes with high
  computational costs. This is a particular issue in video recognition, where
  models are often applied repeatedly across frames or temporal chunks. In this
  work, we exploit temporal redundancy between subsequent inputs to reduce the
  cost of Transformers for video processing. We describe a method for identifying
  and re-processing only those tokens that have changed significantly over time.
  Our proposed family of models, Eventful Transformers, can be converted from
  existing Transformers (often without any re-training) and give adaptive control
  over the compute cost at runtime. We evaluate our method on large-scale
  datasets for video object detection (ImageNet VID) and action recognition
  (EPIC-Kitchens 100). Our approach leads to significant computational savings
  (on the order of 2-4x) with only minor reductions in accuracy.
paper_ai_summary: The proposed Eventful Transformers identify and reprocess only significant changes in video frames to reduce computational costs by 2-4x with minimal accuracy loss.
paper_ai_keywords: Vision Transformers computational costs video recognition temporal redundancy tokens Eventful Transformers adaptive control