Deep Learning, Machine Learning, Self Supervised Learning, Aritificial Intelligence, Neural Networks.
authors propose to store around 3K features, i.e., in the same range as the number of code vectors. This means that they only keep features from the last 15 batches with a batch size of 256, while contrastive methods typically need to store the last 65K instances obtained from the last 250 batches.