Cross Resolution Encoding-Decoding For Detection Transformers
<|reference_start|>Cross Resolution Encoding-Decoding For Detection Transformers: Detection Transformers (DETR) are renowned object detection pipelines, however computationally efficient multiscale detection using DETR is still challenging. In this paper, we propose a Cross-Resolution Encoding-Decoding (CRED) mechanism that allows DETR to achieve the accuracy of high-resolution detection while having the speed of low-resolution detection. CRED is based on two modules; Cross Resolution Attention Module (CRAM) and One Step Multiscale Attention (OSMA). CRAM is designed to transfer the knowledge of low-resolution encoder output to a high-resolution feature. While OSMA is designed to fuse multiscale features in a single step and produce a feature map of a desired resolution enriched with multiscale information. When used in prominent DETR methods, CRED delivers accuracy similar to the high-resolution DETR counterpart in roughly 50% fewer FLOPs. Specifically, state-of-the-art DN-DETR, when used with CRED (calling CRED-DETR), becomes 76% faster, with ~50% reduced FLOPs than its high-resolution counterpart with 202 G FLOPs on MS-COCO benchmark. We plan to release pretrained CRED-DETRs for use by the community. Code: https://github.com/ashishkumar822/CRED-DETR<|reference_end|>
@article{kumar2024cross,
    title={Cross Resolution Encoding-Decoding For Detection Transformers},
    author={Ashish Kumar, Jaesik Park},
    journal={arXiv preprint arXiv:2410.04088},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04088},
    primaryClass={cs.CV}
}