Machine Learning, Artificial Intelligence, Technology, Deep Learning, Programming.
thus can’t be trained in parallel). This gap is what modern LLMs like Mamba and RWKV seek to address. They build RNN-like models that have efficient inference, with a slight twist. Instead of adding non-linearity at every step, they only add non-linearity at the last steps. Since most of the