  Associative memory architectures are designed for memorization but also
offer, through their retrieval method, a form of generalization to unseen
inputs: stored memories can be seen as prototypes from this point of view.
Focusing on Modern Hopfield Networks (MHN), we show that a large memorization
capacity undermines the generalization opportunity. We offer a solution to
better optimize this tradeoff. It relies on Minimum Description Length (MDL) to
determine during training which memories to store, as well as how many of them.

Machine Learning;Computation and Language