BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement
  for Transformers in Large-Scale Time Series Modeling
Time series models face significant challenges in scaling to handle large and
complex datasets, akin to the scaling achieved by large language models (LLMs).
The unique characteristics of time series data and the computational demands of
model scaling necessitate innovative approaches. While researchers have
explored various architectures such as Transformers, LSTMs, and GRUs to address
these challenges, we propose a novel solution using RWKV-7, which incorporates
meta-learning into its state update mechanism. By integrating RWKV-7's time mix
and channel mix components into the transformer-based time series model Timer,
we achieve a substantial performance improvement of approximately 1.13 to 43.3x
and a 4.5x reduction in training time with 1/23 parameters, all while utilizing
fewer parameters. Our code and model weights are publicly available for further
research and development at https://github.com/Alic-Li/BlackGoose_Rimer.
67cfa436d37b8309603da1ee
Li weile
extracted_confirmed
2025-03-11T04:29:57.207Z
66de61d7174e9c6971dbb253
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sM0xfS7HAkf_6GmkEGjDk.png
Alic Li
user
Alic-Li
67cfa436d37b8309603da1ef
Liu Xiao
extracted_confirmed
2025-03-11T03:33:06.087Z
6176b32847ee6431f632981e
https://cdn-avatars.huggingface.co/v1/production/uploads/6176b32847ee6431f632981e/02rZ_oLAI0Ll6Y6be7Q9F.jpeg
Xiaoliu.x
user
xiaol
BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement
  for Transformers in Large-Scale Time Series Modeling
Time series models face significant challenges in scaling to handle large and
complex datasets, akin to the scaling achieved by large language models (LLMs).
The unique characteristics of time series data and the computational demands of
model scaling necessitate innovative approaches. While researchers have
explored various architectures such as Transformers, LSTMs, and GRUs to address
these challenges, we propose a novel solution using RWKV-7, which incorporates
meta-learning into its state update mechanism. By integrating RWKV-7's time mix
and channel mix components into the transformer-based time series model Timer,
we achieve a substantial performance improvement of approximately 1.13 to 43.3x
and a 4.5x reduction in training time with 1/23 parameters, all while utilizing
fewer parameters. Our code and model weights are publicly available for further
research and development at https://github.com/Alic-Li/BlackGoose_Rimer.
A novel approach using RWKV-7 with meta-learning improves the performance of transformer-based time series models, achieving significant reductions in training time and parameter use.
Transformers LSTMs GRUs RWKV-7 meta-learning time mix channel mix Timer performance improvement training time parameter use