JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create
  State-of-the-Art Japanese Retrievers with Constrained Resources
Neural Information Retrieval has advanced rapidly in high-resource languages,
but progress in lower-resource ones such as Japanese has been hindered by data
scarcity, among other challenges. Consequently, multilingual models have
dominated Japanese retrieval, despite their computational inefficiencies and
inability to capture linguistic nuances. While recent multi-vector monolingual
models like JaColBERT have narrowed this gap, they still lag behind
multilingual methods in large-scale evaluations. This work addresses the
suboptimal training methods of multi-vector retrievers in lower-resource
settings, focusing on Japanese. We systematically evaluate and improve key
aspects of the inference and training settings of JaColBERT, and more broadly,
multi-vector models. We further enhance performance through a novel checkpoint
merging step, showcasing it to be an effective way of combining the benefits of
fine-tuning with the generalization capabilities of the original checkpoint.
Building on our analysis, we introduce a novel training recipe, resulting in
the JaColBERTv2.5 model. JaColBERTv2.5, with only 110 million parameters and
trained in under 15 hours on 4 A100 GPUs, significantly outperforms all
existing methods across all common benchmarks, reaching an average score of
0.754, significantly above the previous best of 0.720. To support future
research, we make our final models, intermediate checkpoints and all data used
publicly available.
66a9c341697b8c2e9399a459
Benjamin Clavié
extracted_confirmed
2024-08-18T17:33:47.111Z
5ff60d4352c26e9bc240badd
https://cdn-avatars.huggingface.co/v1/production/uploads/5ff60d4352c26e9bc240badd/HzoknJibrSasc1ZzU71XA.png
Benjamin Clavié
user
bclavie
JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create
  State-of-the-Art Japanese Retrievers with Constrained Resources
Neural Information Retrieval has advanced rapidly in high-resource languages,
but progress in lower-resource ones such as Japanese has been hindered by data
scarcity, among other challenges. Consequently, multilingual models have
dominated Japanese retrieval, despite their computational inefficiencies and
inability to capture linguistic nuances. While recent multi-vector monolingual
models like JaColBERT have narrowed this gap, they still lag behind
multilingual methods in large-scale evaluations. This work addresses the
suboptimal training methods of multi-vector retrievers in lower-resource
settings, focusing on Japanese. We systematically evaluate and improve key
aspects of the inference and training settings of JaColBERT, and more broadly,
multi-vector models. We further enhance performance through a novel checkpoint
merging step, showcasing it to be an effective way of combining the benefits of
fine-tuning with the generalization capabilities of the original checkpoint.
Building on our analysis, we introduce a novel training recipe, resulting in
the JaColBERTv2.5 model. JaColBERTv2.5, with only 110 million parameters and
trained in under 15 hours on 4 A100 GPUs, significantly outperforms all
existing methods across all common benchmarks, reaching an average score of
0.754, significantly above the previous best of 0.720. To support future
research, we make our final models, intermediate checkpoints and all data used
publicly available.
Improvements to the training and inference methods of multi-vector models like JaColBERT enhance performance in Japanese information retrieval, achieving state-of-the-art results.
Neural Information Retrieval JaColBERT multi-vector retrievers checkpoint merging fine-tuning generalization capabilities JaColBERTv2.5