Answers.Body: <p>Translate the image into a description. Add the description of the image with the text.</p>
  <p>Embed both of those so you have the information from the image and the information from the text.</p>
  <p>How does this sound?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-23T11:12:22.857Z
Answers.Text: So with `into a description` you mean the process of my second code example?
Answers.UserDisplayName: Jeanluca Scaljeri
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-25T13:28:01.863Z
Answers.Text: Of course I could add the text in the process of creating the image embedding
Answers.UserDisplayName: Jeanluca Scaljeri
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-23T07:43:01.517Z
Answers.LastActivityDate: 2024-04-23T07:43:01.517Z
Answers.UserDisplayName: Joseph
Body: <p>I'm using Ollama to run llm's. I can create embeddings for text and images, which I store in ChromaDB. The goal of all this, is to find content which best fits a question, so I can create a good answer.</p>
  <p>Here is how I create embeddings for text</p>
  <pre><code>curl http://localhost:11434/api/embeddings -d '{
      &quot;model&quot;: &quot;llama2&quot;,
      &quot;prompt&quot;: &quot;Here is an article about llamas...&quot;
  }'
  </code></pre>
  <p>and for images (base64)</p>
  <pre><code>curl http://localhost:11434/api/generate -d '{
    &quot;model&quot;: &quot;llava&quot;,
    &quot;prompt&quot;:&quot;What is in this picture?&quot;,
    &quot;stream&quot;: false,
    &quot;images&quot;: [&quot;iVBORw0KGgoAAAANS.....]
  }
  </code></pre>
  <p>However, now I have some content which is text and image together. The text (or the image) alone is meaningless, but together they tell a complete story. So my question is how should I handle this? Is it possible to create a single embedding for both?</p>
  
Tags: <large-language-models><vectors><vector-semantics>
Title: How to create 1 embedding for text + image
UserDisplayName: Jeanluca Scaljeri
ThreadText: # How to create 1 embedding for text + image
  
  (Asked by: username_0 on 2024-04-22)
  
  I'm using Ollama to run llm's. I can create embeddings for text and images, which I store in ChromaDB. The goal of all this, is to find content which best fits a question, so I can create a good answer.
  
  Here is how I create embeddings for text
  
  ```
  curl http://localhost:11434/api/embeddings -d '{
      "model": "llama2",
      "prompt": "Here is an article about llamas..."
  }'
  
  ```
  
  and for images (base64)
  
  ```
  curl http://localhost:11434/api/generate -d '{
    "model": "llava",
    "prompt":"What is in this picture?",
    "stream": false,
    "images": ["iVBORw0KGgoAAAANS.....]
  }
  
  ```
  
  However, now I have some content which is text and image together. The text (or the image) alone is meaningless, but together they tell a complete story. So my question is how should I handle this? Is it possible to create a single embedding for both?
  
  **Tags:** large-language-models, vectors, vector-semantics
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-23. Score: 0
  
  Translate the image into a description. Add the description of the image with the text.
  
  Embed both of those so you have the information from the image and the information from the text.
  
  How does this sound?
  
  > **Comments:**
  > **username_0**: So with `into a description` you mean the process of my second code example?  
  > **username_0**: Of course I could add the text in the process of creating the image embedding  
  
  ---
  
  