readme: 
  # mGPT
  
  mGPT is pre-trained on the [mC4 dataset](https://huggingface.co/datasets/mc4) using a causal language modeling objective. It was introduced in this [paper](https://arxiv.org/abs/2110.06609) and first released on this page.
  
  ## Model description
  
  mGPT is a Transformer-based model which pre-trained on massive multilingual data covering over 101 languages. Similar to GPT-2, It was pre-trained on the raw texts only, with no human labeling. We use the same tokenization and vocabulary as the [mT5 model](https://huggingface.co/google/mt5-base).
  
  ## Intended uses
  
  You can use the raw model for text generation or using prompts for adapting it to a downstream task.
  
  ## How to use
  
  You can use this model directly with a pipeline for text generation.  Here is how to use this model to get the features of a given text in PyTorch:
  
  ```python
  from transformers import MT5Tokenizer, GPT2LMHeadModel, TextGenerationPipeline
  
  tokenizer = MT5Tokenizer.from_pretrained("THUMT/mGPT")
  model = GPT2LMHeadModel.from_pretrained("THUMT/mGPT")
  
  pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
  text = "Replace me by any text you'd like."
  text = pipeline(text, do_sample=True,  max_length=1024)[0]["generated_text"]
  ```
  
  ## Preprocessing
  
  The texts are tokenized using `sentencepiece` and a vocabulary size of 250,100. The inputs are sequences of 1,024 consecutive tokens. We use `<extra_id_0>` to separate lines in a document.
  
  ## BibTeX entry and citation info
  
  ```bibtex
  @misc{tan2021msp,
      title={MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators},
      author={Zhixing Tan and Xiangwen Zhang and Shuo Wang and Yang Liu},
      year={2021},
      eprint={2110.06609},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
  }
  ```
  