Getting different result with different batch size and sequence length
Here is the code:
```
import os
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
from transformers import BertTokenizer, BertModel
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained("bert-base-uncased")
model = model.to(device)
model.eval()

text1 = [
    "Replace me by any text you'd like.",
    # "The weather is great!"
]
encoded_input1 = tokenizer(text1, return_tensors='pt', padding=True)
encoded_input1 = encoded_input1.to(device)
with torch.no_grad():
    output1 = model(**encoded_input1).last_hidden_state
cls1 = output1[:, 0, :]

text2 = [
    "Replace me by any text you'd like.",
    "The weather is great!"
    # "The result changed with different batch size and sequence length"
]
encoded_input2 = tokenizer(text2, return_tensors='pt', padding=True)
encoded_input2 = encoded_input2.to(device)
with torch.no_grad():
    output2 = model(**encoded_input2).last_hidden_state
cls2 = output2[:, 0, :]

text3 = [
    "Replace me by any text you'd like.",
    # "The weather is great!"
    "The result is changed with different batch size or sequence length."
]
encoded_input3 = tokenizer(text3, return_tensors='pt', padding=True)
encoded_input3 = encoded_input3.to(device)
with torch.no_grad():
    output3 = model(**encoded_input3).last_hidden_state
cls3 = output3[:, 0, :]

print(torch.equal(cls1[0], cls2[0]))
print(torch.equal(cls1[0], cls3[0]))
print(torch.equal(cls2[0], cls3[0]))
```
All of these results are False. Is it as expected?
Hey @JaheimLee 👋 

Yes, minor fluctuations are to be expected. Their causes include (but are not limited to) the numerical masking from the attention mask and the order of operations in fp32 computations.

Because of these fluctuations, we typically consider results correct if their are within 1e-5 of each other, in examples like yours :)<|||||>Ok，thanks！