Hierarchical State Space Models for Continuous Sequence-to-Sequence
  Modeling
Reasoning from sequences of raw sensory data is a ubiquitous problem across
fields ranging from medical devices to robotics. These problems often involve
using long sequences of raw sensor data (e.g. magnetometers, piezoresistors) to
predict sequences of desirable physical quantities (e.g. force, inertial
measurements). While classical approaches are powerful for locally-linear
prediction problems, they often fall short when using real-world sensors. These
sensors are typically non-linear, are affected by extraneous variables (e.g.
vibration), and exhibit data-dependent drift. For many problems, the prediction
task is exacerbated by small labeled datasets since obtaining ground-truth
labels requires expensive equipment. In this work, we present Hierarchical
State-Space Models (HiSS), a conceptually simple, new technique for continuous
sequential prediction. HiSS stacks structured state-space models on top of each
other to create a temporal hierarchy. Across six real-world sensor datasets,
from tactile-based state prediction to accelerometer-based inertial
measurement, HiSS outperforms state-of-the-art sequence models such as causal
Transformers, LSTMs, S4, and Mamba by at least 23% on MSE. Our experiments
further indicate that HiSS demonstrates efficient scaling to smaller datasets
and is compatible with existing data-filtering techniques. Code, datasets and
videos can be found on https://hiss-csp.github.io.
65cef0dfe5413a96b0884bc9
Raunaq Bhirangi
claimed_verified
2024-02-16T16:20:20.506Z
65cf8a3f00d099dc4cbbb653
/avatars/9c0ff791e679000aa7f446d8f5f0c3e6.svg
Raunaq Bhirangi
user
raunaqbhirangi
65cef0dfe5413a96b0884bca
Chenyu Wang
65cef0dfe5413a96b0884bcb
Venkatesh Pattabiraman
claimed_verified
2024-02-19T09:34:24.821Z
65d12297617d1f7450e7dd06
/avatars/7d02f152fdeed4d5bdebe120f9864794.svg
Venkatesh Pattabiraman
user
venky2k
65cef0dfe5413a96b0884bcc
Carmel Majidi
65cef0dfe5413a96b0884bcd
Abhinav Gupta
65cef0dfe5413a96b0884bce
Tess Hellebrekers
65cef0dfe5413a96b0884bcf
Lerrel Pinto
admin_assigned
2024-02-16T14:32:26.515Z
6388cdb6685efaedd15dc0f3
/avatars/b2b74454bf86bf61fd90ba088644f547.svg
Lerrel Pinto
user
lerrel
Hierarchical State Space Models for Continuous Sequence-to-Sequence
  Modeling
Reasoning from sequences of raw sensory data is a ubiquitous problem across
fields ranging from medical devices to robotics. These problems often involve
using long sequences of raw sensor data (e.g. magnetometers, piezoresistors) to
predict sequences of desirable physical quantities (e.g. force, inertial
measurements). While classical approaches are powerful for locally-linear
prediction problems, they often fall short when using real-world sensors. These
sensors are typically non-linear, are affected by extraneous variables (e.g.
vibration), and exhibit data-dependent drift. For many problems, the prediction
task is exacerbated by small labeled datasets since obtaining ground-truth
labels requires expensive equipment. In this work, we present Hierarchical
State-Space Models (HiSS), a conceptually simple, new technique for continuous
sequential prediction. HiSS stacks structured state-space models on top of each
other to create a temporal hierarchy. Across six real-world sensor datasets,
from tactile-based state prediction to accelerometer-based inertial
measurement, HiSS outperforms state-of-the-art sequence models such as causal
Transformers, LSTMs, S4, and Mamba by at least 23% on MSE. Our experiments
further indicate that HiSS demonstrates efficient scaling to smaller datasets
and is compatible with existing data-filtering techniques. Code, datasets and
videos can be found on https://hiss-csp.github.io.
Hierarchical State-Space Models (HiSS) stack structured state-space models to create a temporal hierarchy, outperforming sequence models like causal Transformers, LSTMs, S4, and Mamba in continuous sequential prediction across real-world sensor datasets.
Hierarchical State-Space Models HiSS state-space models causal Transformers LSTMs S4 Mamba MSE