SODFormer: Streaming Object Detection with Transformer Using Events and Frames.DAVIS camera, streaming two complementary sensing modalities of asynchronous events and frames, has gradually been used to address major object detection challenges (e.g., fast motion blur and low-light).  To address this challenge, we propose a novel streaming object detector with Transformer, namely SODFormer, which first integrates events and frames to continuously detect objects in an asynchronous manner.  The results show that the proposed SODFormer outperforms four state-of-the-art methods and our eight baselines by a significant margin.  Our dataset and code can be available at https://github.com/dianzl/SODFormer.
SODFormer
SODFormer: Streaming Object Detection with Transformer Using Events and Frames.DAVIS camera, streaming two complementary sensing modalities of asynchronous events and frames, has gradually been used to address major object detection challenges (e.g., fast motion blur and low-light). 
DAVIS camera
Technically, we first build a large-scale multimodal neuromorphic object detection dataset (i.e., PKU-DAVIS-SOD) over 1080.1k manual labels. 
PKU-DAVIS-SOD
SODFormer: Streaming Object Detection with Transformer Using Events and Frames.DAVIS camera, streaming two complementary sensing modalities of asynchronous events and frames, has gradually been used to address major object detection challenges (e.g., fast motion blur and low-light).  To address this challenge, we propose a novel streaming object detector with Transformer, namely SODFormer, which first integrates events and frames to continuously detect objects in an asynchronous manner.  Then, we design a spatiotemporal Transformer architecture to detect objects via an end-to-end sequence prediction problem, where the novel temporal Transformer module leverages rich temporal cues from two visual streams to improve the detection performance. 
Transformer
Our dataset and code can be available at https://github.com/dianzl/SODFormer.
GitHub