Audio Encoder (ESC) Depth Encoder (SUN)

Image Encoder ViT-S
52.8
54.8

ViT-B
ViT-H

ViT-B
56.7
60.3

ViT-S
30.7
33.3

ViT-B
26.7
29.5

Table 6. Capacity of the audio and depth encoders and their
impact on performance. A stronger image encoder improves per-
formance for both audio and depth tasks. As the number of (image,
depth) pairs is small, a smaller encoder improves performance for
depth. For audio classification, a larger encoder is better.

Batch size → 512
47.3
39.4

NYU-D

ESC

1k
46.5
53.9

2k
43.0
56.7

4k
39.9
53.9

Table 7. Effect of scaling batch size. We found the optimal batch
size for contrastive loss varied by the modality. For image-depth
task, a smaller batch size was better, likely due to the small size
and limited diversity of the original dataset. For audio-video task
where we have a lot more positive and negative audio-video pairs,
using a large batch size lead to better results.
IMAGEBIND- One Embedding Space To Bind Them A