Language Model, Transformers, Deep Learning, Artificial Intelligence, Machine Learning.
like the value vector as in the attention. Then it projects v, x_1, ..., x_n with learnable filters h_1, ..., h_n. Hyena applies a multiplicative gating interaction to the projected vectors, similar to LSTMs. This gating is used to control the information flow through the recurrence. Hyena uses an