Fill-Mask Pipeline BERT on a sentence ending with/without a dot
## Environment info
- `transformers` version: 4.8.1
- Platform: Windows 10
- Python version: 3.7.10
- PyTorch version: 1.9.0

## Problem
Why do the following sentences give different predicted words?
Sentence 1: [MASK] is the capital of France
Sentence 2: [MASK] is the capital of France.
where the second sentence is with a dot as the ending.

## Program
from transformers import pipeline

fillmask_bert = pipeline('fill-mask', model="bert-base-uncased")
pred_1 = fillmask_bert("[MASK] is the capital of France")
for item in pred_1:
    print(item)
print()

pred_2 = fillmask_bert("[MASK] is the capital of France.")
for item in pred_2:
    print(item)
print()

## Fill-Mask Results
"[MASK] is the capital of France"
{'sequence': 'it is the capital of france', 'score': 0.33803579211235046, 'token': 2009, 'token_str': 'it'}
{'sequence': 'paris is the capital of france', 'score': 0.30488064885139465, 'token': 3000, 'token_str': 'paris'}
{'sequence': 'toulouse is the capital of france', 'score': 0.04690197855234146, 'token': 17209, 'token_str': 'toulouse'}
{'sequence': 'lyon is the capital of france', 'score': 0.02363533526659012, 'token': 10241, 'token_str': 'lyon'}
{'sequence': 'marseille is the capital of france', 'score': 0.022472301498055458, 'token': 16766, 'token_str': 'marseille'}

"[MASK] is the capital of France."
{'sequence': 'paris is the capital of france.', 'score': 0.6210566163063049, 'token': 3000, 'token_str': 'paris'}
{'sequence': 'it is the capital of france.', 'score': 0.07383635640144348, 'token': 2009, 'token_str': 'it'}
{'sequence': 'toulouse is the capital of france.', 'score': 0.036227378994226456, 'token': 17209, 'token_str': 'toulouse'}
{'sequence': 'marseille is the capital of france.', 'score': 0.031809303909540176, 'token': 16766, 'token_str': 'marseille'}
{'sequence': 'lyon is the capital of france.', 'score': 0.03100072219967842, 'token': 10241, 'token_str': 'lyon'}



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.