scale up the model size to 175B. Oguz et al. [45] performs domain-matched pre-training to improve
in-domain results. SPAR [11] trains a dense retriever by treating BM25 as a teacher model. Although
the aforementioned approaches can easily obtain abundant supervision signals, such synthetic data
tend to be of low quality. Results on the BEIR benchmark [53] show they struggle to match the
performance of BM25 if not further ﬁne-tuned on labeled datasets.
Evaluation and interpretation of text embeddings are also non-trivial. Most benchmarks measure the
embedding quality through downstream task performances. For example, SentEval [13] uses linear
probing and a collection of semantic textual similarity (STS) datasets, while the BEIR benchmark
[53] focuses on zero-shot information retrieval scenarios. The recently introduced MTEB benchmark
[40] combines 56 datasets spanning across 8 tasks and 112 languages. Experiments show no model
E5