Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception
<|reference_start|>Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception: Real-time object detection is critical for the decision-making process for many real-world applications, such as collision avoidance and path planning in autonomous driving. This work presents an innovative real-time streaming perception method, Transtreaming, which addresses the challenge of real-time object detection with dynamic computational delay. The core innovation of Transtreaming lies in its adaptive delay-aware transformer, which can concurrently predict multiple future frames and select the output that best matches the real-world present time, compensating for any system-induced computation delays. The proposed model outperforms the existing state-of-the-art methods, even in single-frame detection scenarios, by leveraging a transformer-based methodology. It demonstrates robust performance across a range of devices, from powerful V100 to modest 2080Ti, achieving the highest level of perceptual accuracy on all platforms. Unlike most state-of-the-art methods that struggle to complete computation within a single frame on less powerful devices, Transtreaming meets the stringent real-time processing requirements on all kinds of devices. The experimental results emphasize the system's adaptability and its potential to significantly improve the safety and reliability for many real-world systems, such as autonomous driving.<|reference_end|>
@article{zhang2024transtreaming:,
    title={Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming
  Perception},
    author={Xiang Zhang, Yufei Cui, Chenchen Fu, Weiwei Wu, Zihao Wang, Yuyang
  Sun, Xue Liu},
    journal={arXiv preprint arXiv:2409.06584},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.06584},
    primaryClass={cs.CV}
}