Answers.Body: <p>Do you mean sending the vector of embedded text as the context, instead of the text itself?</p>
  <p>If you think it through that might mean sending in <code>[2.12234, 12.134123, 3.132412, ....., 1.123124]</code> to chatGPT instead of <code>Hi, can you tell me what is the grandfather paradox?</code>. Do you spot the problem here?</p>
  <p>GPT models are designed in a way to only take textual inputs and spit out textual inputs. So, in short, what you are proposing is not possible with the API endpoints currently provided by OpenAI.</p>
  <p>Theoretically, taking in an embedding vector and producing the relevant text is a possibility but considering the transformer architecture, there is a logical fallacy in your proposed argument.</p>
  <ol>
  <li>Embeddings are produced from the text itself, so to infer a sequence from a generated embedding would mean that you have already used up a certain toke length, i.e. at this point it really doesn't matter what is the max allowed token length.</li>
  </ol>
  <p>Not just that, but the ADA embedding model is different and is fine-tuned to generate better similarity rather than aid in generating CHATesque text. So, even if OpenAI offered that possibility (which it doesn't) the generated text wouldn't be as good as sending in text as context to GPT4 api endpoint.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-11T19:10:24.913Z
Answers.LastActivityDate: 2023-07-11T19:10:24.913Z
Answers.UserDisplayName: Chinmay
Answers.Body: <p>Vectors are not provided as context - you need to provide the original text. Embeddings are used with e.g. vector databased to provide similarity queries so you can reduce the context you provide to GPT by only selecting relevant parts (e.g. 1-2 documents from a library of 1000s of documents).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T18:18:32.903Z
Answers.Text: Pleast break the answer into smaller bits for unexperienced questioners. It will increase your answer's effectiveness.
Answers.UserDisplayName: Chinmay
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-11T09:51:33.760Z
Answers.LastActivityDate: 2023-07-11T09:51:33.760Z
Answers.UserDisplayName: larsmoa
Body: <p>Since the context/memory of a chat or question for LLMs more precisely GPT is limited to a token length I struggle about how to provide own data that the model got not trained on.
  A very common approach looks like embeddings are the way to.</p>
  <p>OpenAI provided an article <a href="https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb" rel="nofollow noreferrer">https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb</a>
  how to create an embedding of a user query, match it against a local vector database ans provide the closest results <strong>as text</strong> to the context/memory.</p>
  <p>Here I do struggle, since it might be very well possible that even we find the most matching documents locally in a vector database, context might still be too small if we would like to provide multiple matches.</p>
  <p>The question to me is, how could I send all the relevant embedding vectors rather than the relevant texts which got matched to the vectors?
  These vectors are highly condensed and would save a lot of tokens. GPT would anyhow be able to understand the vector since they created it from their embeddings API, right?</p>
  <p>Or is it just not possible to convert the vector back to text at their end?</p>
  
Tags: <gpt><embeddings>
Title: How can I send vectors as a chat context?
UserDisplayName: dc10
ThreadText: # How can I send vectors as a chat context?
  
  (Asked by: username_0 on 2023-05-11)
  
  Since the context/memory of a chat or question for LLMs more precisely GPT is limited to a token length I struggle about how to provide own data that the model got not trained on.
  A very common approach looks like embeddings are the way to.
  
  OpenAI provided an article <https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb>
  how to create an embedding of a user query, match it against a local vector database ans provide the closest results **as text** to the context/memory.
  
  Here I do struggle, since it might be very well possible that even we find the most matching documents locally in a vector database, context might still be too small if we would like to provide multiple matches.
  
  The question to me is, how could I send all the relevant embedding vectors rather than the relevant texts which got matched to the vectors?
  These vectors are highly condensed and would save a lot of tokens. GPT would anyhow be able to understand the vector since they created it from their embeddings API, right?
  
  Or is it just not possible to convert the vector back to text at their end?
  
  **Tags:** gpt, embeddings
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-07-11. Score: 2
  
  Do you mean sending the vector of embedded text as the context, instead of the text itself?
  
  If you think it through that might mean sending in `[2.12234, 12.134123, 3.132412, ....., 1.123124]` to chatGPT instead of `Hi, can you tell me what is the grandfather paradox?`. Do you spot the problem here?
  
  GPT models are designed in a way to only take textual inputs and spit out textual inputs. So, in short, what you are proposing is not possible with the API endpoints currently provided by OpenAI.
  
  Theoretically, taking in an embedding vector and producing the relevant text is a possibility but considering the transformer architecture, there is a logical fallacy in your proposed argument.
  
  1. Embeddings are produced from the text itself, so to infer a sequence from a generated embedding would mean that you have already used up a certain toke length, i.e. at this point it really doesn't matter what is the max allowed token length.
  
  Not just that, but the ADA embedding model is different and is fine-tuned to generate better similarity rather than aid in generating CHATesque text. So, even if OpenAI offered that possibility (which it doesn't) the generated text wouldn't be as good as sending in text as context to GPT4 api endpoint.
  
  
  ---
  
  ### Answer by username_2 on 2023-07-11. Score: 0
  
  Vectors are not provided as context - you need to provide the original text. Embeddings are used with e.g. vector databased to provide similarity queries so you can reduce the context you provide to GPT by only selecting relevant parts (e.g. 1-2 documents from a library of 1000s of documents).
  
  > **Comments:**
  > **username_1**: Pleast break the answer into smaller bits for unexperienced questioners. It will increase your answer's effectiveness.  
  
  ---
  
  