Answers.Body: <p>One can use the <a href="https://huggingface.co/papers/2309.09400" rel="nofollow noreferrer">CulturaX</a> dataset (<a href="https://arxiv.org/pdf/2309.09400.pdf" rel="nofollow noreferrer">paper</a>) to train an LLM in another language. It contains 27 TB of texts divided into 167 languages.</p>
  <p>Otherwise, one can use <a href="https://github.com/nlp-uoregon/Okapi" rel="nofollow noreferrer">Okapi</a> (<a href="https://arxiv.org/pdf/2307.16039.pdf" rel="nofollow noreferrer">paper</a>):</p>
  <blockquote>
  <p>Okapi Resources: We provide resources to perform instruction tuning with RLHF for 26 languages, including ChatGPT prompts, multilingual instruction datasets and multilingual response ranking data.</p>
  <p>Okapi Models: We provide RLHF-based instruction-tuned LLMs for 26 languages on Okapi dataset. Our models include both BLOOM-based and LLaMa-based versions. We also provide scripts to interact with our models and fine-tune LLMs with our resources.</p>
  <p>Multilingual Evaluation Benchmark Datasets: We provide three benchmark datasets for evaluating Multilingual Large Language Models (LLMs) for 26 languages. You can access the full datasets and evaluation scripts: <a href="https://github.com/nlp-uoregon/mlmm-evaluation" rel="nofollow noreferrer">here</a>.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-06T13:42:01.150Z
Answers.LastActivityDate: 2024-03-06T13:42:01.150Z
Answers.UserDisplayName: Franck Dernoncourt
Body: <p>I have sentence data in a language that is not widely in use and as such popular LLMs do not support the language. I want to train some language model such that given some question, it is able to respond back in the same language, just as in ChatGPT just with a different language.</p>
  <p>In such a case, what language model is publically available and is sufficiently powerful? Or would it be possible to use popular LLMs (such as ChatGPT) to achieve such a goal?</p>
  
Tags: <natural-language-processing><large-language-models><chatgpt><language-model>
Title: publically available language models that can be used to train arbitrary language data?
UserDisplayName: Neijal Kanderbalt
ThreadText: # publically available language models that can be used to train arbitrary language data?
  
  (Asked by: username_0 on 2024-01-12)
  
  I have sentence data in a language that is not widely in use and as such popular LLMs do not support the language. I want to train some language model such that given some question, it is able to respond back in the same language, just as in ChatGPT just with a different language.
  
  In such a case, what language model is publically available and is sufficiently powerful? Or would it be possible to use popular LLMs (such as ChatGPT) to achieve such a goal?
  
  **Tags:** natural-language-processing, large-language-models, chatgpt, language-model
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-03-06. Score: 0
  
  One can use the [CulturaX](https://huggingface.co/papers/2309.09400) dataset ([paper](https://arxiv.org/pdf/2309.09400.pdf)) to train an LLM in another language. It contains 27 TB of texts divided into 167 languages.
  
  Otherwise, one can use [Okapi](https://github.com/nlp-uoregon/Okapi) ([paper](https://arxiv.org/pdf/2307.16039.pdf)):
  
  > Okapi Resources: We provide resources to perform instruction tuning with RLHF for 26 languages, including ChatGPT prompts, multilingual instruction datasets and multilingual response ranking data.
  >
  > Okapi Models: We provide RLHF-based instruction-tuned LLMs for 26 languages on Okapi dataset. Our models include both BLOOM-based and LLaMa-based versions. We also provide scripts to interact with our models and fine-tune LLMs with our resources.
  >
  > Multilingual Evaluation Benchmark Datasets: We provide three benchmark datasets for evaluating Multilingual Large Language Models (LLMs) for 26 languages. You can access the full datasets and evaluation scripts: [here](https://github.com/nlp-uoregon/mlmm-evaluation).
  
  
  ---
  
  