visual clips from YouTube accompanied by textual descrip-
tions. It consists of clips from the Audioset dataset as de-
scribed earlier. We use the splits as provided in [52],1 which
removes clips that overlap with the VGGSound dataset. We
end up with 48198 training, 418 validation and 796 test
clips. We only use the test set for zero-shot evaluation of
our model. The task is text→audio retrieval, and evaluation
is performed using recall@K.
VGGSound (VGGS) [8]. This dataset contains about 200K
video clips of 10s length, annotated with 309 sound classes
consisting of human actions, sound-emitting objects and
human-object interactions. We only use the audio from the
test set (with 14073 clips) for 0-shot classification. The
evaluation is done using the top-1 accuracy metric.
SUN RGB-D (SUN). We use the registered RGB and Depth
maps provided in the SUN RGB-D [67] dataset train set
(∼5K pairs) for training our model. We follow [20] to post
IMAGEBIND- One Embedding Space To Bind Them A