Microsoft Corporation

https://github.com/microsoft/unilm

Abstract

This paper presents E5 1, a family of state-of-the-art text embeddings that transfer
well to a wide range of tasks. The model is trained in a contrastive manner with
weak supervision signals from our curated large-scale text pair dataset (called
CCPairs). E5 can be readily used as a general-purpose embedding model for any
tasks requiring a single-vector representation of texts such as retrieval, clustering,
and classiﬁcation, achieving strong performance in both zero-shot and ﬁne-tuned
settings. We conduct extensive evaluations on 56 datasets from the BEIR and
MTEB benchmarks. For zero-shot settings, E5 is the ﬁrst model that outperforms
the strong BM25 baseline on the BEIR retrieval benchmark without using any
labeled data. When ﬁne-tuned, E5 obtains the best results on the MTEB benchmark,
beating existing embedding models with 40× more parameters.

1

Introduction
E5