Body: <p>Training of LLM aka GPT models is clear on how is trained but can't find any info how is &quot;mapped&quot; query to internal tokens and generates response tokens more precise inference phase which consists of natural language processing and dialog management, with focus on how user input is computed to generate response tokens/content(dialog management part would be perfect but would work a general GPT inference too), mathematics and phases/steps are similar to traditional NLP models (matrix of vectors mapped to vocal vector)? Checked more resources but none provides a clear picture on how do work internally <a href="https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/tokens" rel="nofollow noreferrer">tokens</a>, <a href="https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/what-runs-chatgpt-inside-microsoft-s-ai-supercomputer-featuring/ba-p/3830281" rel="nofollow noreferrer">chatGPT</a> and <a href="https://openai.com/blog/chatgpt" rel="nofollow noreferrer">OpenAI chatGPT</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-13T18:12:05.637Z
Comments.Text: Would you add more explain of "mapping" and "generating back"? Are you looking for Positional Encoding? Or is the mapping you mentioned that some functions at Tokenize data section at https://keras.io/examples/generative/text_generation_gpt/?
Comments.UserDisplayName: Cloud Cho
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-13T19:45:14.553Z
Comments.Text: Is more how user input is actually computed to generate response, kinda response back, how is computed (matrix and vectors in distributed manner to tind starting of sequence and keep generating based on tokens and then a sum of product on response to generate desired length token response) ?
Comments.UserDisplayName: n1tk
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-13T21:40:36.133Z
Comments.Text: So you're looking inferencing (not training)?
Comments.UserDisplayName: Cloud Cho
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-14T01:28:03.180Z
Comments.Text: Exactly, and in the question I do point out that training is very clear mathematically for me but can’t find math explanation related to inference phase which consists of natural language processing and dialog management …
Comments.UserDisplayName: n1tk
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-16T00:23:28.690Z
Comments.Text: Is this what you looking for https://pi-tau.github.io/posts/transformer/#inference?
Comments.UserDisplayName: Cloud Cho
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-11-15T02:43:02.197Z
Comments.Text: Unfortunately not, looking for a math description on prompt engineering for the LLM models, how is query tokens “mapped” to an answer from the model.
Comments.UserDisplayName: n1tk
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-11-15T18:53:52.920Z
Comments.Text: If you are looking mathematical approach token mapping, it is related position in sentence. Here are two reference from spaCy Transition/Dependency Parser (https://spacy.io/api/dependencyparser) and Mathew's explanation on the Parser with number (https://explosion.ai/blog/parsing-english-in-python#example-usage)
Comments.UserDisplayName: Cloud Cho
Tags: <chatgpt><gpt><question-answering>
Title: How does query of LLM/GPT models work?
UserDisplayName: n1tk
ThreadText: # How does query of LLM/GPT models work?
  
  (Asked by: username_0 on 2023-09-13)
  
  Training of LLM aka GPT models is clear on how is trained but can't find any info how is "mapped" query to internal tokens and generates response tokens more precise inference phase which consists of natural language processing and dialog management, with focus on how user input is computed to generate response tokens/content(dialog management part would be perfect but would work a general GPT inference too), mathematics and phases/steps are similar to traditional NLP models (matrix of vectors mapped to vocal vector)? Checked more resources but none provides a clear picture on how do work internally [tokens](https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/tokens), [chatGPT](https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/what-runs-chatgpt-inside-microsoft-s-ai-supercomputer-featuring/ba-p/3830281) and [OpenAI chatGPT](https://openai.com/blog/chatgpt)
  
  **Tags:** chatgpt, gpt, question-answering
  
  > **Comments:**
  > **username_1**: Would you add more explain of "mapping" and "generating back"? Are you looking for Positional Encoding? Or is the mapping you mentioned that some functions at Tokenize data section at https://keras.io/examples/generative/text_generation_gpt/?  
  > **username_0**: Is more how user input is actually computed to generate response, kinda response back, how is computed (matrix and vectors in distributed manner to tind starting of sequence and keep generating based on tokens and then a sum of product on response to generate desired length token response) ?  
  > **username_1**: So you're looking inferencing (not training)?  
  > **username_0**: Exactly, and in the question I do point out that training is very clear mathematically for me but can’t find math explanation related to inference phase which consists of natural language processing and dialog management …  
  > **username_1**: Is this what you looking for https://pi-tau.github.io/posts/transformer/#inference?  
  > **username_0**: Unfortunately not, looking for a math description on prompt engineering for the LLM models, how is query tokens “mapped” to an answer from the model.  
  > **username_1**: If you are looking mathematical approach token mapping, it is related position in sentence. Here are two reference from spaCy Transition/Dependency Parser (https://spacy.io/api/dependencyparser) and Mathew's explanation on the Parser with number (https://explosion.ai/blog/parsing-english-in-python#example-usage)  
  
  ---
  
  ## 0 answers
  
  