Reducing the Footprint of Multi-Vector Retrieval with Minimal
  Performance Impact via Token Pooling
Over the last few years, multi-vector retrieval methods, spearheaded by
ColBERT, have become an increasingly popular approach to Neural IR. By storing
representations at the token level rather than at the document level, these
methods have demonstrated very strong retrieval performance, especially in
out-of-domain settings. However, the storage and memory requirements necessary
to store the large number of associated vectors remain an important drawback,
hindering practical adoption. In this paper, we introduce a simple
clustering-based token pooling approach to aggressively reduce the number of
vectors that need to be stored. This method can reduce the space & memory
footprint of ColBERT indexes by 50% with virtually no retrieval performance
degradation. This method also allows for further reductions, reducing the
vector count by 66%-to-75% , with degradation remaining below 5% on a vast
majority of datasets. Importantly, this approach requires no architectural
change nor query-time processing, and can be used as a simple drop-in during
indexation with any ColBERT-like model.
66f64f2b8f96964e4b024e82
Benjamin Clavié
claimed_verified
2024-11-27T22:20:59.550Z
5ff60d4352c26e9bc240badd
https://cdn-avatars.huggingface.co/v1/production/uploads/5ff60d4352c26e9bc240badd/HzoknJibrSasc1ZzU71XA.png
Benjamin Clavié
user
bclavie
66f64f2b8f96964e4b024e83
Antoine Chaffin
claimed_verified
2024-09-27T13:48:00.209Z
609bbe2f4932693ca2009d6a
https://cdn-avatars.huggingface.co/v1/production/uploads/1620819560688-609bbe2f4932693ca2009d6a.jpeg
Antoine Chaffin
user
NohTow
66f64f2b8f96964e4b024e84
Griffin Adams
Reducing the Footprint of Multi-Vector Retrieval with Minimal
  Performance Impact via Token Pooling
Over the last few years, multi-vector retrieval methods, spearheaded by
ColBERT, have become an increasingly popular approach to Neural IR. By storing
representations at the token level rather than at the document level, these
methods have demonstrated very strong retrieval performance, especially in
out-of-domain settings. However, the storage and memory requirements necessary
to store the large number of associated vectors remain an important drawback,
hindering practical adoption. In this paper, we introduce a simple
clustering-based token pooling approach to aggressively reduce the number of
vectors that need to be stored. This method can reduce the space & memory
footprint of ColBERT indexes by 50% with virtually no retrieval performance
degradation. This method also allows for further reductions, reducing the
vector count by 66%-to-75% , with degradation remaining below 5% on a vast
majority of datasets. Importantly, this approach requires no architectural
change nor query-time processing, and can be used as a simple drop-in during
indexation with any ColBERT-like model.
A clustering-based token pooling method significantly reduces the storage and memory requirements of ColBERT indexes without compromising retrieval performance.
multi-vector retrieval ColBERT token level document level retrieval performance vector pooling clustering-based pooling ColBERT indexes query-time processing