title: JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create
    State-of-the-Art Japanese Retrievers with Constrained Resources
summary: Neural Information Retrieval has advanced rapidly in high-resource languages,
  but progress in lower-resource ones such as Japanese has been hindered by data
  scarcity, among other challenges. Consequently, multilingual models have
  dominated Japanese retrieval, despite their computational inefficiencies and
  inability to capture linguistic nuances. While recent multi-vector monolingual
  models like JaColBERT have narrowed this gap, they still lag behind
  multilingual methods in large-scale evaluations. This work addresses the
  suboptimal training methods of multi-vector retrievers in lower-resource
  settings, focusing on Japanese. We systematically evaluate and improve key
  aspects of the inference and training settings of JaColBERT, and more broadly,
  multi-vector models. We further enhance performance through a novel checkpoint
  merging step, showcasing it to be an effective way of combining the benefits of
  fine-tuning with the generalization capabilities of the original checkpoint.
  Building on our analysis, we introduce a novel training recipe, resulting in
  the JaColBERTv2.5 model. JaColBERTv2.5, with only 110 million parameters and
  trained in under 15 hours on 4 A100 GPUs, significantly outperforms all
  existing methods across all common benchmarks, reaching an average score of
  0.754, significantly above the previous best of 0.720. To support future
  research, we make our final models, intermediate checkpoints and all data used
  publicly available.
paper_authors._id: 66a9c341697b8c2e9399a459
paper_authors.name: Benjamin Clavié
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-18T17:33:47.111Z
paper_authors._id: 5ff60d4352c26e9bc240badd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5ff60d4352c26e9bc240badd/HzoknJibrSasc1ZzU71XA.png
paper_authors.fullname: Benjamin Clavié
paper_authors.type: user
paper_authors.user: bclavie
paper_title: JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create
    State-of-the-Art Japanese Retrievers with Constrained Resources
paper_summary: Neural Information Retrieval has advanced rapidly in high-resource languages,
  but progress in lower-resource ones such as Japanese has been hindered by data
  scarcity, among other challenges. Consequently, multilingual models have
  dominated Japanese retrieval, despite their computational inefficiencies and
  inability to capture linguistic nuances. While recent multi-vector monolingual
  models like JaColBERT have narrowed this gap, they still lag behind
  multilingual methods in large-scale evaluations. This work addresses the
  suboptimal training methods of multi-vector retrievers in lower-resource
  settings, focusing on Japanese. We systematically evaluate and improve key
  aspects of the inference and training settings of JaColBERT, and more broadly,
  multi-vector models. We further enhance performance through a novel checkpoint
  merging step, showcasing it to be an effective way of combining the benefits of
  fine-tuning with the generalization capabilities of the original checkpoint.
  Building on our analysis, we introduce a novel training recipe, resulting in
  the JaColBERTv2.5 model. JaColBERTv2.5, with only 110 million parameters and
  trained in under 15 hours on 4 A100 GPUs, significantly outperforms all
  existing methods across all common benchmarks, reaching an average score of
  0.754, significantly above the previous best of 0.720. To support future
  research, we make our final models, intermediate checkpoints and all data used
  publicly available.
paper_ai_keywords: multilingual models computational inefficiencies linguistic nuances multi-vector monolingual models JaColBERT inference settings training settings checkpoint merging fine-tuning generalization capabilities training recipe JaColBERTv2.5 A100 GPUs