Answers.Body: <p>I've tried Mistral-7B-Instruct-v0.2, and it worked out of the box for your example, as did Qwen1.5-1.5b. However, Qwen1.5-0.5b had issues, but I think with enough training, you can achieve decent results.</p>
  <p>I believe the size/quality trade-off of starting from 1.5b and lowering it could deteriorate in corner cases, but the only way to find out is through experimentation with validation.</p>
  <p>One note is that you can generate almost an infinite amount of data by removing the JSON structure. 15k seems sufficient if you have just a product JSONs, but for more diversity, you might want more data. The SFT stage in an LLM usually takes from 200k up to a million examples.</p>
  <p>You can also constrain the output using a scheme by restricting the output tokens. See one example of such an approach at <a href="https://github.com/1rgs/jsonformer" rel="nofollow noreferrer">https://github.com/1rgs/jsonformer</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-21T08:35:19.710Z
Answers.LastActivityDate: 2024-02-21T08:35:19.710Z
Answers.UserDisplayName: Kirill Fedyanin
Body: <p>I have tried training T5-small, T5-base and T5-Large on around 15K rows of data where input data was something like but I did not get desired results</p>
  <pre><code>Nutrition Facts,
  100g per,
  Energy 646.95Kcal Carbohydrates 19.31g,
   Protein 21.94g 53.55g Total Fat 6.64g Saturated Fat 14.97g Dietary Fiber,
  &lt;1.Omg Cholesterol Sodium 0.29g Sugars 3.39g,
  Lightly Salted and to Perfection,
  Ingredients: Peanuts, Almonds,,
  Cashews, Pistachios, Vegetable Oil, Salt,
  aa, ,
  74G,
  Pistachio, 61129110611336177
  WE ARE NUTS ABOUT QUALITY,
  Baked,
  Nuts Salted,
  Mixed
  WE ARE NUTS ABOUT,
  Community 364, 13 Street Plot No. 36,
  Al Area 1, 24149,UAE 4971 4 3355777,
  License Number: 224614 VAT No: 100058529700003,
  CERTIFIED COMPANY,
  ALLERGEN WARNING: in a facility that also processes nuts, sesame and mustard,
  Store in a cool dry place away from heat moisture,
  Instruction Once store in airtight container and consume before expiry date,
  Pro: 14/12/23,
  Exp:13/12/24,
  Net 40gms
  </code></pre>
  <p>Output data will be in JSON format of the above details.
  What language models can be trained for this purpose and minimum how many Parameters it should have?</p>
  
Tags: <natural-language-processing><language-model><natural-language-understanding>
Title: What language model can convert normal text to JSON data
UserDisplayName: Arjun Singh
ThreadText: # What language model can convert normal text to JSON data
  
  (Asked by: username_0 on 2024-02-20)
  
  I have tried training T5-small, T5-base and T5-Large on around 15K rows of data where input data was something like but I did not get desired results
  Nutrition Facts,
  100g per,
  Energy 646.95Kcal Carbohydrates 19.31g,
   Protein 21.94g 53.55g Total Fat 6.64g Saturated Fat 14.97g Dietary Fiber,
  <1.Omg Cholesterol Sodium 0.29g Sugars 3.39g,
  Lightly Salted and to Perfection,
  Ingredients: Peanuts, Almonds,,
  Cashews, Pistachios, Vegetable Oil, Salt,
  aa, ,
  74G,
  Pistachio, 61129110611336177
  WE ARE NUTS ABOUT QUALITY,
  Baked,
  Nuts Salted,
  Mixed
  WE ARE NUTS ABOUT,
  Community 364, 13 Street Plot No. 36,
  Al Area 1, 24149,UAE 4971 4 3355777,
  License Number: 224614 VAT No: 100058529700003,
  CERTIFIED COMPANY,
  ALLERGEN WARNING: in a facility that also processes nuts, sesame and mustard,
  Store in a cool dry place away from heat moisture,
  Instruction Once store in airtight container and consume before expiry date,
  Pro: 14/12/23,
  Exp:13/12/24,
  Net 40gms
  
  Output data will be in JSON format of the above details.
  What language models can be trained for this purpose and minimum how many Parameters it should have?
  
  **Tags:** natural-language-processing, language-model, natural-language-understanding
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-02-21. Score: 1 | Accepted answer
  
  I've tried Mistral-7B-Instruct-v0.2, and it worked out of the box for your example, as did Qwen1.5-1.5b. However, Qwen1.5-0.5b had issues, but I think with enough training, you can achieve decent results.
  
  I believe the size/quality trade-off of starting from 1.5b and lowering it could deteriorate in corner cases, but the only way to find out is through experimentation with validation.
  
  One note is that you can generate almost an infinite amount of data by removing the JSON structure. 15k seems sufficient if you have just a product JSONs, but for more diversity, you might want more data. The SFT stage in an LLM usually takes from 200k up to a million examples.
  
  You can also constrain the output using a scheme by restricting the output tokens. See one example of such an approach at <https://github.com/1rgs/jsonformer>.
  
  
  ---
  
  