Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
Transformer-level quality while reducing required compute in training by 20% at a sequence length of 2K. Notably, Hyena operators are also faster, offering twice the efficiency of highly optimized attention operators. Hyena first projects the input into a set of vectors v, x_1, ..., x_n and v acts