MambaEVT: Event Stream based Visual Object Tracking using State Space
  Model
Event camera-based visual tracking has drawn more and more attention in
recent years due to the unique imaging principle and advantages of low energy
consumption, high dynamic range, and dense temporal resolution. Current
event-based tracking algorithms are gradually hitting their performance
bottlenecks, due to the utilization of vision Transformer and the static
template for target object localization. In this paper, we propose a novel
Mamba-based visual tracking framework that adopts the state space model with
linear complexity as a backbone network. The search regions and target template
are fed into the vision Mamba network for simultaneous feature extraction and
interaction. The output tokens of search regions will be fed into the tracking
head for target localization. More importantly, we consider introducing a
dynamic template update strategy into the tracking framework using the Memory
Mamba network. By considering the diversity of samples in the target template
library and making appropriate adjustments to the template memory module, a
more effective dynamic template can be integrated. The effective combination of
dynamic and static templates allows our Mamba-based tracking algorithm to
achieve a good balance between accuracy and computational cost on multiple
large-scale datasets, including EventVOT, VisEvent, and FE240hz. The source
code will be released on https://github.com/Event-AHU/MambaEVT
66c550cc3496dcc9c2769d7a
Xiao Wang
66c550cc3496dcc9c2769d7b
Chao wang
66c550cc3496dcc9c2769d7c
Shiao Wang
66c550cc3496dcc9c2769d7d
Xixi Wang
66c550cc3496dcc9c2769d7e
Zhicheng Zhao
66c550cc3496dcc9c2769d7f
Lin Zhu
66c550cc3496dcc9c2769d80
Bo Jiang
MambaEVT: Event Stream based Visual Object Tracking using State Space
  Model
Event camera-based visual tracking has drawn more and more attention in
recent years due to the unique imaging principle and advantages of low energy
consumption, high dynamic range, and dense temporal resolution. Current
event-based tracking algorithms are gradually hitting their performance
bottlenecks, due to the utilization of vision Transformer and the static
template for target object localization. In this paper, we propose a novel
Mamba-based visual tracking framework that adopts the state space model with
linear complexity as a backbone network. The search regions and target template
are fed into the vision Mamba network for simultaneous feature extraction and
interaction. The output tokens of search regions will be fed into the tracking
head for target localization. More importantly, we consider introducing a
dynamic template update strategy into the tracking framework using the Memory
Mamba network. By considering the diversity of samples in the target template
library and making appropriate adjustments to the template memory module, a
more effective dynamic template can be integrated. The effective combination of
dynamic and static templates allows our Mamba-based tracking algorithm to
achieve a good balance between accuracy and computational cost on multiple
large-scale datasets, including EventVOT, VisEvent, and FE240hz. The source
code will be released on https://github.com/Event-AHU/MambaEVT
A new Mamba-based tracking framework combines static and dynamic templates with a vision Mamba network to improve accuracy and computational efficiency in event camera-based visual tracking.
event-based tracking vision Transformer state space model feature extraction interaction output tokens tracking head target localization dynamic template Memory Mamba network template memory module EventVOT VisEvent FE240hz