  previous section as well). One of the languages is selected for each training sample, and the model input is a
  sentence of 256 tokens, that may span over several documents in one of those languages.
Masked language modeling (MLM) which is like RoBERTa. One of the languages is selected for each training sample,
  and the model input is a sentence of 256 tokens, that may span over several documents in one of those languages,
  with dynamic masking of the tokens.
A combination of MLM and translation language modeling (TLM). This consists of concatenating a sentence in two
  different languages, with random masking. To predict one of the masked tokens, the model can use both, the
  surrounding context in language 1 and the context given by language 2.
Checkpoints refer to which method was used for pretraining by having clm, mlm or mlm-tlm in their names. On top
