title: Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary
    Detection
summary: Open-vocabulary detection (OVD) aims to detect objects beyond a predefined
  set of categories. As a pioneering model incorporating the YOLO series into
  OVD, YOLO-World is well-suited for scenarios prioritizing speed and
  efficiency.However, its performance is hindered by its neck feature fusion
  mechanism, which causes the quadratic complexity and the limited guided
  receptive fields.To address these limitations, we present Mamba-YOLO-World, a
  novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation
  Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce
  an innovative State Space Model-based feature fusion mechanism consisting of a
  Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan
  algorithm with linear complexity and globally guided receptive fields. It
  leverages multi-modal input sequences and mamba hidden states to guide the
  selective scanning process.Experiments demonstrate that our model outperforms
  the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and
  fine-tuning settings while maintaining comparable parameters and FLOPs.
  Additionally, it surpasses existing state-of-the-art OVD methods with fewer
  parameters and FLOPs.
paper_authors._id: 66e82f60e6c22fa18351df70
paper_authors.name: Haoxuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-26T08:25:43.075Z
paper_authors._id: 66f4c39e379dab9895cd29ab
paper_authors.avatarUrl: /avatars/4bbda793d5939db3b4610d8151c34a10.svg
paper_authors.fullname: Haoxuan Wang
paper_authors.type: user
paper_authors.user: Xuan-World
paper_authors._id: 66e82f60e6c22fa18351df71
paper_authors.name: Qingdong He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-17T07:52:40.358Z
paper_authors._id: 64799de9587656b38e8dcf7a
paper_authors.avatarUrl: /avatars/87069f03d44cc2d9e0458bc17bd0159a.svg
paper_authors.fullname: heqingdong
paper_authors.type: user
paper_authors.user: hqd
paper_authors._id: 66e82f60e6c22fa18351df72
paper_authors.name: Jinlong Peng
paper_authors._id: 66e82f60e6c22fa18351df73
paper_authors.name: Hao Yang
paper_authors._id: 66e82f60e6c22fa18351df74
paper_authors.name: Mingmin Chi
paper_authors._id: 66e82f60e6c22fa18351df75
paper_authors.name: Yabiao Wang
paper_title: Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary
    Detection
paper_summary: Open-vocabulary detection (OVD) aims to detect objects beyond a predefined
  set of categories. As a pioneering model incorporating the YOLO series into
  OVD, YOLO-World is well-suited for scenarios prioritizing speed and
  efficiency.However, its performance is hindered by its neck feature fusion
  mechanism, which causes the quadratic complexity and the limited guided
  receptive fields.To address these limitations, we present Mamba-YOLO-World, a
  novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation
  Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce
  an innovative State Space Model-based feature fusion mechanism consisting of a
  Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan
  algorithm with linear complexity and globally guided receptive fields. It
  leverages multi-modal input sequences and mamba hidden states to guide the
  selective scanning process.Experiments demonstrate that our model outperforms
  the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and
  fine-tuning settings while maintaining comparable parameters and FLOPs.
  Additionally, it surpasses existing state-of-the-art OVD methods with fewer
  parameters and FLOPs.
paper_ai_summary: Mamba-YOLO-World improves open-vocabulary detection by introducing a linear complexity MambaFusion-PAN neck architecture, enhancing performance on COCO and LVIS benchmarks.
paper_ai_keywords: open-vocabulary detection OVD YOLO series YOLO-World neck feature fusion quadratic complexity guided receptive fields MambaFusion Path Aggregation Network MambaFusion-PAN State Space Model Parallel-Guided Selective Scan Serial-Guided Selective Scan multi-modal input sequences mamba hidden states selective scanning process