is right?
https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py
in 225 row

![image](https://user-images.githubusercontent.com/20811685/74525332-d1480e00-4f5b-11ea-8918-ae22507de8e8.png)

# 10% of the time, we replace masked input tokens with random word
but write 0.5 
is ok?
Hi @ARDUJS can you update your issue title to something more descriptive? Thanks!<|||||>Should be correct -> 80% masked, that means 20% is left. Using this 20% in 50 % the random word is used, 50% original token is kept. So both random word and original has an overall prob. of 10%.

Original BERT is using the same logic, see [here](https://github.com/google-research/bert/blob/master/create_pretraining_data.py#L391).