title: LUSIFER: Language Universal Space Integration for Enhanced Multilingual
    Embeddings with Large Language Models
summary: Recent advancements in large language models (LLMs) based embedding models
  have established new state-of-the-art benchmarks for text embedding tasks,
  particularly in dense vector-based retrieval. However, these models
  predominantly focus on English, leaving multilingual embedding capabilities
  largely unexplored. To address this limitation, we present LUSIFER, a novel
  zero-shot approach that adapts LLM-based embedding models for multilingual
  tasks without requiring multilingual supervision. LUSIFER's architecture
  combines a multilingual encoder, serving as a language-universal learner, with
  an LLM-based embedding model optimized for embedding-specific tasks. These
  components are seamlessly integrated through a minimal set of trainable
  parameters that act as a connector, effectively transferring the multilingual
  encoder's language understanding capabilities to the specialized embedding
  model. Additionally, to comprehensively evaluate multilingual embedding
  performance, we introduce a new benchmark encompassing 5 primary embedding
  tasks, 123 diverse datasets, and coverage across 14 languages. Extensive
  experimental results demonstrate that LUSIFER significantly enhances the
  multilingual performance across various embedding tasks, particularly for
  medium and low-resource languages, without requiring explicit multilingual
  training data.
paper_authors._id: 677b6abf6b1653e3951b4cbd
paper_authors.name: Hieu Man
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:35:03.741Z
paper_authors._id: 6225ce77b213da14d165fa50
paper_authors.avatarUrl: /avatars/64d2307859d73343280a895ca81e3136.svg
paper_authors.fullname: Hieu Man
paper_authors.type: user
paper_authors.user: Hieuman
paper_authors._id: 677b6abf6b1653e3951b4cbe
paper_authors.name: Nghia Trung Ngo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:35:09.619Z
paper_authors._id: 658a8208c4b2004663d82daf
paper_authors.avatarUrl: /avatars/af6bcee06aec82602c2b931f79c008e7.svg
paper_authors.fullname: Nghia Trung Ngo
paper_authors.type: user
paper_authors.user: ntnghia1811
paper_authors._id: 677b6abf6b1653e3951b4cbf
paper_authors.name: Viet Dac Lai
paper_authors._id: 677b6abf6b1653e3951b4cc0
paper_authors.name: Ryan A. Rossi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:35:19.050Z
paper_authors._id: 62a3ab83e4dd6252344d27cd
paper_authors.avatarUrl: /avatars/7ca8510f70a58dc207b104240e30c35c.svg
paper_authors.fullname: Ryan A. Rossi
paper_authors.type: user
paper_authors.user: ryanrossi
paper_authors._id: 677b6abf6b1653e3951b4cc1
paper_authors.name: Franck Dernoncourt
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:35:26.065Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 677b6abf6b1653e3951b4cc2
paper_authors.name: Thien Huu Nguyen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:35:31.396Z
paper_authors._id: 64804fad8c6a3b8f11f73912
paper_authors.avatarUrl: /avatars/61e37a91d4bba35fda9bf52aadd87745.svg
paper_authors.fullname: Thien Huu Nguyen
paper_authors.type: user
paper_authors.user: anoperson
paper_title: LUSIFER: Language Universal Space Integration for Enhanced Multilingual
    Embeddings with Large Language Models
paper_summary: Recent advancements in large language models (LLMs) based embedding models
  have established new state-of-the-art benchmarks for text embedding tasks,
  particularly in dense vector-based retrieval. However, these models
  predominantly focus on English, leaving multilingual embedding capabilities
  largely unexplored. To address this limitation, we present LUSIFER, a novel
  zero-shot approach that adapts LLM-based embedding models for multilingual
  tasks without requiring multilingual supervision. LUSIFER's architecture
  combines a multilingual encoder, serving as a language-universal learner, with
  an LLM-based embedding model optimized for embedding-specific tasks. These
  components are seamlessly integrated through a minimal set of trainable
  parameters that act as a connector, effectively transferring the multilingual
  encoder's language understanding capabilities to the specialized embedding
  model. Additionally, to comprehensively evaluate multilingual embedding
  performance, we introduce a new benchmark encompassing 5 primary embedding
  tasks, 123 diverse datasets, and coverage across 14 languages. Extensive
  experimental results demonstrate that LUSIFER significantly enhances the
  multilingual performance across various embedding tasks, particularly for
  medium and low-resource languages, without requiring explicit multilingual
  training data.
paper_ai_keywords: zero-shot approach multilingual encoder language-universal learner LLM-based embedding model trainable parameters connector multilingual embedding tasks benchmark dense vector-based retrieval