Answers.Body: <p>you mean the whole corpus or just a chat inside a corpus?
  a simple chat on slack from the start to end usually shouldn't be too long. of your whole data could be large.</p>
  <p>To (re-)train your model, you need to split your data into some chunks. length of each chunks could be e.g 4k to 16k token depending on the (maximum) context size of your pretrained LLM.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-09T13:24:17.493Z
Answers.LastActivityDate: 2024-07-09T13:24:17.493Z
Answers.UserDisplayName: saleh
Body: <p>Given a corpus of data like a log of Slack conversations, I want to be able to use this and generate fictitious conversations e.g., given 10 conversations, I want to be able to scale this up to 100 conversations. Is there a toolkit to do this? I tried searching but could not find anything. I understand the most naive approach would be to use a pre-trained LLM, give the corpus to it as input and ask it to do the job. I haven't tried this but the real corpus will be large enough such that it won't fit in context window of a pre-trained LLM.</p>
  
Tags: <natural-language-processing><large-language-models>
Title: How to generate synthetic text data for LLM fine tuning?
UserDisplayName: morpheus
ThreadText: # How to generate synthetic text data for LLM fine tuning?
  
  (Asked by: username_0 on 2024-07-05)
  
  Given a corpus of data like a log of Slack conversations, I want to be able to use this and generate fictitious conversations e.g., given 10 conversations, I want to be able to scale this up to 100 conversations. Is there a toolkit to do this? I tried searching but could not find anything. I understand the most naive approach would be to use a pre-trained LLM, give the corpus to it as input and ask it to do the job. I haven't tried this but the real corpus will be large enough such that it won't fit in context window of a pre-trained LLM.
  
  **Tags:** natural-language-processing, large-language-models
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-07-09. Score: 0
  
  you mean the whole corpus or just a chat inside a corpus?
  a simple chat on slack from the start to end usually shouldn't be too long. of your whole data could be large.
  
  To (re-)train your model, you need to split your data into some chunks. length of each chunks could be e.g 4k to 16k token depending on the (maximum) context size of your pretrained LLM.
  
  
  ---
  
  