LUSIFER: Language Universal Space Integration for Enhanced Multilingual
  Embeddings with Large Language Models
Recent advancements in large language models (LLMs) based embedding models
have established new state-of-the-art benchmarks for text embedding tasks,
particularly in dense vector-based retrieval. However, these models
predominantly focus on English, leaving multilingual embedding capabilities
largely unexplored. To address this limitation, we present LUSIFER, a novel
zero-shot approach that adapts LLM-based embedding models for multilingual
tasks without requiring multilingual supervision. LUSIFER's architecture
combines a multilingual encoder, serving as a language-universal learner, with
an LLM-based embedding model optimized for embedding-specific tasks. These
components are seamlessly integrated through a minimal set of trainable
parameters that act as a connector, effectively transferring the multilingual
encoder's language understanding capabilities to the specialized embedding
model. Additionally, to comprehensively evaluate multilingual embedding
performance, we introduce a new benchmark encompassing 5 primary embedding
tasks, 123 diverse datasets, and coverage across 14 languages. Extensive
experimental results demonstrate that LUSIFER significantly enhances the
multilingual performance across various embedding tasks, particularly for
medium and low-resource languages, without requiring explicit multilingual
training data.
677b6abf6b1653e3951b4cbd
Hieu Man
admin_assigned
2025-01-06T08:35:03.741Z
6225ce77b213da14d165fa50
/avatars/64d2307859d73343280a895ca81e3136.svg
Hieu Man
user
Hieuman
677b6abf6b1653e3951b4cbe
Nghia Trung Ngo
admin_assigned
2025-01-06T08:35:09.619Z
658a8208c4b2004663d82daf
/avatars/af6bcee06aec82602c2b931f79c008e7.svg
Nghia Trung Ngo
user
ntnghia1811
677b6abf6b1653e3951b4cbf
Viet Dac Lai
677b6abf6b1653e3951b4cc0
Ryan A. Rossi
admin_assigned
2025-01-06T08:35:19.050Z
62a3ab83e4dd6252344d27cd
/avatars/7ca8510f70a58dc207b104240e30c35c.svg
Ryan A. Rossi
user
ryanrossi
677b6abf6b1653e3951b4cc1
Franck Dernoncourt
admin_assigned
2025-01-06T08:35:26.065Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
677b6abf6b1653e3951b4cc2
Thien Huu Nguyen
admin_assigned
2025-01-06T08:35:31.396Z
64804fad8c6a3b8f11f73912
/avatars/61e37a91d4bba35fda9bf52aadd87745.svg
Thien Huu Nguyen
user
anoperson
LUSIFER: Language Universal Space Integration for Enhanced Multilingual
  Embeddings with Large Language Models
Recent advancements in large language models (LLMs) based embedding models
have established new state-of-the-art benchmarks for text embedding tasks,
particularly in dense vector-based retrieval. However, these models
predominantly focus on English, leaving multilingual embedding capabilities
largely unexplored. To address this limitation, we present LUSIFER, a novel
zero-shot approach that adapts LLM-based embedding models for multilingual
tasks without requiring multilingual supervision. LUSIFER's architecture
combines a multilingual encoder, serving as a language-universal learner, with
an LLM-based embedding model optimized for embedding-specific tasks. These
components are seamlessly integrated through a minimal set of trainable
parameters that act as a connector, effectively transferring the multilingual
encoder's language understanding capabilities to the specialized embedding
model. Additionally, to comprehensively evaluate multilingual embedding
performance, we introduce a new benchmark encompassing 5 primary embedding
tasks, 123 diverse datasets, and coverage across 14 languages. Extensive
experimental results demonstrate that LUSIFER significantly enhances the
multilingual performance across various embedding tasks, particularly for
medium and low-resource languages, without requiring explicit multilingual
training data.
LUSIFER, a zero-shot approach using a multilingual encoder and LLM-based embedding model, enhances multilingual performance in text embedding tasks without multilingual supervision.
zero-shot approach multilingual encoder LLM-based embedding model multilingual supervision embedding-specific tasks trainable parameters multilingual performance medium and low-resource languages