5. Ablation Study

We investigate various design choices for learning a joint
embedding space for different modalities. Since the abla-
tion experimental setup is similar to § 4, we only note the
main differences (full details in Appendix C). We report re-
sults on the ESC fold-1 for the ablation study. We use a ViT-
B encoder for the image, audio, depth, and thermal modali-
ties by default and train them for 16 epochs (vs. 32 epochs

ClapsChurch BellsChirping birdsThunderstormDog barkingKeyboard typingClock alarmSea wavesViT-B ViT-L

ViT-H

ViT-B ViT-L ViT-H

50
48
46
44

62

60

58

-

D
U
Y
N

P
I
V
L
L

64

62

1
-
d
l
o
F
C
S
E

24
22
20
18

D
4
o
g
E

ViT-B ViT-L

ViT-H

ViT-B ViT-L

ViT-H
IMAGEBIND- One Embedding Space To Bind Them A