5.5 Analysis

In this section, we conduct a series of analyses to examine various design choices. All the numbers in
this section are from base-size models. For the BEIR benchmark, we choose 6 datasets with more
stable results across different runs. Some negative results are also listed in Appendix C.
Impacts of Batch Size Since we use in-batch negatives for contrastive pre-training, larger batch size
will provide more negatives and therefore improve the quality of the learned text embeddings. In

7https://huggingface.co/spaces/mteb/leaderboard, as of November 22, 2022

7

Table 5: Impacts of different batch sizes for contrastive pre-training.

batch size NFCorpus NQ FiQA Quora DBPedia
32k
8k
1k

39.0
38.5
33.1

40.0
37.6
30.4

35.4
34.0
30.1

35.8
33.3
28.2

85.7
85.7
84.0

Scifact Avg
51.6
73.7
50.2
71.8
69.1
45.8
E5