Flash STU: Fast Spectral Transform Units
<|reference_start|>Flash STU: Fast Spectral Transform Units: This paper describes an efficient, open source PyTorch implementation of the Spectral Transform Unit. We investigate sequence prediction tasks over several modalities including language, robotics, and simulated dynamical systems. We find that for the same parameter count, the STU and its variants outperform the Transformer as well as other leading state space models across various modalities.<|reference_end|>
@article{liu2024flash,
    title={Flash STU: Fast Spectral Transform Units},
    author={Y. Isabel Liu and Windsor Nguyen and Yagiz Devre and Evan Dogariu and
  Anirudha Majumdar and Elad Hazan},
    journal={arXiv preprint arXiv:2409.10489},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.10489},
    primaryClass={cs.LG cs.AI}
}