HRSAM: Efficiently Segment Anything in High-Resolution Images
<|reference_start|>HRSAM: Efficiently Segment Anything in High-Resolution Images: The Segment Anything Model (SAM) has significantly advanced interactive segmentation but struggles with high-resolution images crucial for high-precision segmentation. This is primarily due to the quadratic space complexity of SAM-implemented attention and the length extrapolation issue in common global attention. This study proposes HRSAM that integrates Flash Attention and incorporates Plain, Shifted and newly proposed Cycle-scan Window (PSCWin) attention to address these issues. The shifted window attention is redesigned with padding to maintain consistent window sizes, enabling effective length extrapolation. The cycle-scan window attention adopts the recently developed State Space Models (SSMs) to ensure global information exchange with minimal computational overhead. Such window-based attention allows HRSAM to perform effective attention computations on scaled input images while maintaining low latency. Moreover, we further propose HRSAM++ that additionally employs a multi-scale strategy to enhance HRSAM's performance. The experiments on the high-precision segmentation datasets HQSeg44K and DAVIS show that high-resolution inputs enable the SAM-distilled HRSAM models to outperform the teacher model while maintaining lower latency. Compared to the SOTAs, HRSAM achieves a 1.56 improvement in interactive segmentation's NoC95 metric with only 31% of the latency. HRSAM++ further enhances the performance, achieving a 1.63 improvement in NoC95 with just 38% of the latency.<|reference_end|>
@article{huang2024hrsam:,
    title={HRSAM: Efficiently Segment Anything in High-Resolution Images},
    author={You Huang, Wenbin Lai, Jiayi Ji, Liujuan Cao, Shengchuan Zhang,
  Rongrong Ji},
    journal={arXiv preprint arXiv:2407.02109},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02109},
    primaryClass={cs.CV cs.AI}
}