Hi,
In the roberta paper, the model is trained by dynamic masking of sentences.
If a roberta model is further pre-trained using the run_mlm.py , is the sentences going to be dynamically masked during pre-training using this file or is it statically masked like vanilla BERT?
The script is found here
  
      

      github.com
  

  
    transformers/examples/pytorch/language-modeling at master ·... 9

  master/examples/pytorch/language-modeling

  🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX. - transformers/examples/pytorch/language-modeling at master · huggingface/transformers