You’re in the right direction regarding the features.
My first try would be to extract the image feature vector and use it (instead of the text embeddings) shown in this tutorial: Semantic Search with Approximate Nearest Neighbors and Text Embeddings 2