GATE: General Arabic Text Embedding for Enhanced Semantic Textual
  Similarity with Matryoshka Representation Learning and Hybrid Loss Training
Semantic textual similarity (STS) is a critical task in natural language
processing (NLP), enabling applications in retrieval, clustering, and
understanding semantic relationships between texts. However, research in this
area for the Arabic language remains limited due to the lack of high-quality
datasets and pre-trained models. This scarcity of resources has restricted the
accurate evaluation and advance of semantic similarity in Arabic text. This
paper introduces General Arabic Text Embedding (GATE) models that achieve
state-of-the-art performance on the Semantic Textual Similarity task within the
MTEB benchmark. GATE leverages Matryoshka Representation Learning and a hybrid
loss training approach with Arabic triplet datasets for Natural Language
Inference, which are essential for enhancing model performance in tasks that
demand fine-grained semantic understanding. GATE outperforms larger models,
including OpenAI, with a 20-25% performance improvement on STS benchmarks,
effectively capturing the unique semantic nuances of Arabic.
683d7ba30ba4502a904432b3
Omer Nacar
claimed_verified
2025-06-03T08:49:36.827Z
628f7a71dd993507cfcbe587
https://cdn-avatars.huggingface.co/v1/production/uploads/628f7a71dd993507cfcbe587/3frAcMgfBCx-OJsrdAhnb.png
Omartificial Intelligence Space
user
Omartificial-Intelligence-Space
683d7ba30ba4502a904432b4
Anis Koubaa
683d7ba30ba4502a904432b5
Serry Sibaee
683d7ba30ba4502a904432b6
Yasser Al-Habashi
683d7ba30ba4502a904432b7
Adel Ammar
683d7ba30ba4502a904432b8
Wadii Boulila
GATE: General Arabic Text Embedding for Enhanced Semantic Textual
  Similarity with Matryoshka Representation Learning and Hybrid Loss Training
Semantic textual similarity (STS) is a critical task in natural language
processing (NLP), enabling applications in retrieval, clustering, and
understanding semantic relationships between texts. However, research in this
area for the Arabic language remains limited due to the lack of high-quality
datasets and pre-trained models. This scarcity of resources has restricted the
accurate evaluation and advance of semantic similarity in Arabic text. This
paper introduces General Arabic Text Embedding (GATE) models that achieve
state-of-the-art performance on the Semantic Textual Similarity task within the
MTEB benchmark. GATE leverages Matryoshka Representation Learning and a hybrid
loss training approach with Arabic triplet datasets for Natural Language
Inference, which are essential for enhancing model performance in tasks that
demand fine-grained semantic understanding. GATE outperforms larger models,
including OpenAI, with a 20-25% performance improvement on STS benchmarks,
effectively capturing the unique semantic nuances of Arabic.
GATE models using Matryoshka Representation Learning and a hybrid loss approach achieve state-of-the-art performance on Arabic Semantic Textual Similarity benchmarks.
Matryoshka Representation Learning hybrid loss training triplet datasets Natural Language Inference Semantic Textual Similarity STS benchmarks