Artificial Intelligence, Thesequence, Llm, Machine Learning.
value, and query states from standard attention for long-term memory storage and retrieval. Instead of discarding old KV states, they are stored in the compressive memory and retrieved using attention query states when processing future sequences. The final contextual output combines long-term