from text prompts, the image generation model relies on
text embeddings produced by the pre-trained CLIP-L/14
text encoder. Since IMAGEBIND naturally aligns CLIP’s-
embedding space to that of other modalities proposed in the
paper, we can upgrade the DALLE-2 model to generate im-
ages by prompting it with these new unseen modalities. We
achieve zero-shot audio to image generation with DALLE-2
by simply using the temperature-scaled audio embeddings
generated by IMAGEBIND’s audio encoder as a proxy for
the CLIP’s text embeddings in the DALLE-2’s image gen-
eration model.
Detecting objects using audio. We extract all audio de-
scriptors from the validation set of ESC using an IMAGE-
BIND ViT-B/32 encoder, yielding 400 descriptors in total.
We use an off-the-shelf CLIP-based Detic [86] model and
IMAGEBIND- One Embedding Space To Bind Them A