Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary
  Detection
Open-vocabulary detection (OVD) aims to detect objects beyond a predefined
set of categories. As a pioneering model incorporating the YOLO series into
OVD, YOLO-World is well-suited for scenarios prioritizing speed and
efficiency.However, its performance is hindered by its neck feature fusion
mechanism, which causes the quadratic complexity and the limited guided
receptive fields.To address these limitations, we present Mamba-YOLO-World, a
novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation
Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce
an innovative State Space Model-based feature fusion mechanism consisting of a
Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan
algorithm with linear complexity and globally guided receptive fields. It
leverages multi-modal input sequences and mamba hidden states to guide the
selective scanning process.Experiments demonstrate that our model outperforms
the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and
fine-tuning settings while maintaining comparable parameters and FLOPs.
Additionally, it surpasses existing state-of-the-art OVD methods with fewer
parameters and FLOPs.
66e82f60e6c22fa18351df70
Haoxuan Wang
admin_assigned
2024-09-26T08:25:43.075Z
66f4c39e379dab9895cd29ab
/avatars/4bbda793d5939db3b4610d8151c34a10.svg
Haoxuan Wang
user
Xuan-World
66e82f60e6c22fa18351df71
Qingdong He
admin_assigned
2024-09-17T07:52:40.358Z
64799de9587656b38e8dcf7a
/avatars/87069f03d44cc2d9e0458bc17bd0159a.svg
heqingdong
user
hqd
66e82f60e6c22fa18351df72
Jinlong Peng
66e82f60e6c22fa18351df73
Hao Yang
66e82f60e6c22fa18351df74
Mingmin Chi
66e82f60e6c22fa18351df75
Yabiao Wang
Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary
  Detection
Open-vocabulary detection (OVD) aims to detect objects beyond a predefined
set of categories. As a pioneering model incorporating the YOLO series into
OVD, YOLO-World is well-suited for scenarios prioritizing speed and
efficiency.However, its performance is hindered by its neck feature fusion
mechanism, which causes the quadratic complexity and the limited guided
receptive fields.To address these limitations, we present Mamba-YOLO-World, a
novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation
Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce
an innovative State Space Model-based feature fusion mechanism consisting of a
Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan
algorithm with linear complexity and globally guided receptive fields. It
leverages multi-modal input sequences and mamba hidden states to guide the
selective scanning process.Experiments demonstrate that our model outperforms
the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and
fine-tuning settings while maintaining comparable parameters and FLOPs.
Additionally, it surpasses existing state-of-the-art OVD methods with fewer
parameters and FLOPs.
Mamba-YOLO-World improves open-vocabulary detection by introducing a linear complexity MambaFusion-PAN neck architecture, enhancing performance on COCO and LVIS benchmarks.
open-vocabulary detection OVD YOLO series YOLO-World neck feature fusion quadratic complexity guided receptive fields MambaFusion Path Aggregation Network MambaFusion-PAN State Space Model Parallel-Guided Selective Scan Serial-Guided Selective Scan multi-modal input sequences mamba hidden states selective scanning process