how does masked_lm_labels  work ?
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 
Hi all
from hugging face{https://huggingface.co/transformers/model_doc/bert.html#bertformaskedlm}

in this code :
{
from transformers import BertTokenizer, BertForMaskedLM
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMaskedLM.from_pretrained('bert-base-uncased')

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0) # Batch size 1
outputs = model(input_ids, masked_lm_labels=input_ids)

loss, prediction_scores = outputs[:2]
}

what happens by {outputs = model(input_ids, masked_lm_labels=input_ids)} ?

it will automatically makes [mask] 15% of all the tokens in each sentence of each bs , and calculates loss just for them ?
@thomwolf @tholor<|||||>No I don't think so, you need to mask the tokens and then pass them to model, look here (e.g. evaluate) for an example
https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py<|||||>@blackcat84 , thanks it helped a lot<|||||>@blackcat84 

and one more thing ,
does any function in those scripts , concatenate the short lines to each other ?
in order not to be enforced to pad each line so much<|||||>It's been a while so I might be wrong but I think you are correct, I don't remember in which function though. A simple way to be sure about it is to pass a dummy input to the script/function and check it by yourself<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
