Answers.Body: <p>If you have an OpenAI Pro account you can define your own GPT that can do certain operations.</p>
  <p>If your stuff is publicly accessible, this is one way to get started quickly:</p>
  <ul>
  <li>Upload the documentation files (or your code) to a public gitlab/github repository accessible from the public internet.</li>
  <li>Study the github/gitlab/... REST API , select endpoints that you might need to call in order to answer your questions about the docs under version control. (e.g. /commits endpoint, /issues endpoint etc)</li>
  <li>Define a GPT action that let your GPT (and thus GPT-4o) call those REST API endpoints.</li>
  <li>Open the ChatGPT interface and &quot;start talking with your documents&quot; , leveraging   the power of GPT-4o or GPT-4.</li>
  </ul>
  <p>(Costs for tokens consumed may apply)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-21T08:29:16.557Z
Answers.LastActivityDate: 2024-05-21T08:29:16.557Z
Answers.UserDisplayName: knb
Body: <p>Would it be possible to train my own LLM on a smaller corpus of text, lets say some coding documentation that I then want to ask questions about using the model?</p>
  <hr />
  <p>If so, are there any recommended ways of doing this, i.e is there a prebuilt architecture or library I can use, and just provide the corpus of text.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-07T12:57:57.807Z
Comments.Text: By training to you mean to train from scratch or to fine-tune a pretrained model? The latter is certainly easier.
Comments.UserDisplayName: BanDoP
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-07T17:18:06.393Z
Comments.Text: If the latter works I can try that, do you know how I can go about doing that?
Comments.UserDisplayName: Dylan Dijk
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-07T17:22:20.550Z
Comments.Text: You could start from one of the models that is available at Huggingface.
Comments.UserDisplayName: BanDoP
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-07T17:23:46.430Z
Comments.Text: Ok thanks, any you recommend, and what is the general work flow to tune such a pretrained model.
Comments.UserDisplayName: Dylan Dijk
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-08T11:04:35.333Z
Comments.Text: @BanDoP I found this [article](https://www.datacamp.com/tutorial/fine-tuning-large-language-models), this seems like what I should aim to copy. In particular, fine tuning LLMs for text classification.
Comments.UserDisplayName: Dylan Dijk
Tags: <natural-language-processing><transformer><large-language-models><open-ai><huggingface>
Title: Train my own LLM on a smaller corpus of text?
UserDisplayName: Dylan Dijk
ThreadText: # Train my own LLM on a smaller corpus of text?
  
  (Asked by: username_0 on 2024-04-07)
  
  Would it be possible to train my own LLM on a smaller corpus of text, lets say some coding documentation that I then want to ask questions about using the model?
  
  ---
  
  If so, are there any recommended ways of doing this, i.e is there a prebuilt architecture or library I can use, and just provide the corpus of text.
  
  **Tags:** natural-language-processing, transformer, large-language-models, open-ai, huggingface
  
  > **Comments:**
  > **username_1**: By training to you mean to train from scratch or to fine-tune a pretrained model? The latter is certainly easier.  
  > **username_0**: If the latter works I can try that, do you know how I can go about doing that?  
  > **username_1**: You could start from one of the models that is available at Huggingface.  
  > **username_0**: Ok thanks, any you recommend, and what is the general work flow to tune such a pretrained model.  
  > **username_0**: @username_1 I found this [article](https://www.datacamp.com/tutorial/fine-tuning-large-language-models), this seems like what I should aim to copy. In particular, fine tuning LLMs for text classification.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-05-21. Score: 0
  
  If you have an OpenAI Pro account you can define your own GPT that can do certain operations.
  
  If your stuff is publicly accessible, this is one way to get started quickly:
  
  * Upload the documentation files (or your code) to a public gitlab/github repository accessible from the public internet.
  * Study the github/gitlab/... REST API , select endpoints that you might need to call in order to answer your questions about the docs under version control. (e.g. /commits endpoint, /issues endpoint etc)
  * Define a GPT action that let your GPT (and thus GPT-4o) call those REST API endpoints.
  * Open the ChatGPT interface and "start talking with your documents" , leveraging the power of GPT-4o or GPT-4.
  
  (Costs for tokens consumed may apply)
  
  
  ---
  
  