Audio Mamba: Bidirectional State Space Model for Audio Representation
  Learning
Transformers have rapidly become the preferred choice for audio
classification, surpassing methods based on CNNs. However, Audio Spectrogram
Transformers (ASTs) exhibit quadratic scaling due to self-attention. The
removal of this quadratic self-attention cost presents an appealing direction.
Recently, state space models (SSMs), such as Mamba, have demonstrated potential
in language and vision tasks in this regard. In this study, we explore whether
reliance on self-attention is necessary for audio classification tasks. By
introducing Audio Mamba (AuM), the first self-attention-free, purely SSM-based
model for audio classification, we aim to address this question. We evaluate
AuM on various audio datasets - comprising six different benchmarks - where it
achieves comparable or better performance compared to well-established AST
model.
66612ef00dbf4ebf86c08924
Mehmet Hamza Erol
admin_assigned
2024-06-06T08:47:35.852Z
6404a5c40ab5e22719f3e105
/avatars/f683bd957502162bad10039c28c8c244.svg
Mehmet Hamza Erol
user
mhamzaerol
66612ef00dbf4ebf86c08925
Arda Senocak
admin_assigned
2024-06-06T08:47:43.691Z
658a6098d1331d552b293e58
/avatars/bb06fc9a47cb78ca9b4e9e7da5cc6b58.svg
Arda Senocak
user
ardasnck
66612ef00dbf4ebf86c08926
Jiu Feng
66612ef00dbf4ebf86c08927
Joon Son Chung
admin_assigned
2024-06-06T08:48:07.919Z
647dcc87e4d52fe0e01c8ffc
/avatars/56fbf5604dc4082b0c494bd49f345c71.svg
Joon Son Chung
user
joonson
Audio Mamba: Bidirectional State Space Model for Audio Representation
  Learning
Transformers have rapidly become the preferred choice for audio
classification, surpassing methods based on CNNs. However, Audio Spectrogram
Transformers (ASTs) exhibit quadratic scaling due to self-attention. The
removal of this quadratic self-attention cost presents an appealing direction.
Recently, state space models (SSMs), such as Mamba, have demonstrated potential
in language and vision tasks in this regard. In this study, we explore whether
reliance on self-attention is necessary for audio classification tasks. By
introducing Audio Mamba (AuM), the first self-attention-free, purely SSM-based
model for audio classification, we aim to address this question. We evaluate
AuM on various audio datasets - comprising six different benchmarks - where it
achieves comparable or better performance compared to well-established AST
model.
AuM, a self-attention-free, state space model-based audio classifier, achieves performance comparable to or better than AST models, overcoming quadratic scaling issues.
Audio Spectrogram Transformers (ASTs) self-attention quadratic scaling state state models (SSMs) Audio Mamba (AuM)