The accuracy of test set is different in training and evaluating Bert
Hi,friends, I meet a problem which puzzled me for couple of days.

I want to training a Bert for two classification task on my custom data.
I choose BertForSequenceClassification model.
And I use the 'Trainer' from 'transformers', the tokenizer, model, metrics and Trainer code as follows:
`


tokenizer=BertTokenizerFast.from_pretrained("bert_base_chinese")

model = BertForSequenceClassification.from_pretrained("bert-base-chinese", return_dict=True)


def computed_metric(eval_pred):

    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)

    precision, recall, f1, _ =precision_recall_fscore_support(labels,predictions, average="macro")

    acc = accuracy_score(labels, predictions)

    return {"accuracy": acc, 'precision: ':precision, "f1" : f1, "recall" : recall }

trainer = Trainer(
    model,
    args,
    train_dataset= dataset_train,
    eval_dataset= dataset_test,
    tokenizer= tokenizer,
    compute_metrics= computed_metric,

)
`

The problem is: in the training process "eval_accuracy: 0.82", however,  after t he training, I tested the model separately, and the statistical accuracy was only 0.74.
The code for the test as follows：
`
def tokenize(x):

    return tokenizer([x[0]], [x[1]], truncation=True, padding='max_length',max_length=512)


answer_model = []  #record the answer from model

model.eval()

for i in range(len(data_text)):

    encoded_input = tokenize(data_text[i])

    encoded_input["input_ids"] = torch.tensor(encoded_input["input_ids"])
    encoded_input["token_type_ids"] = torch.tensor(encoded_input["token_type_ids"])
    encoded_input["attention_mask"] = torch.tensor(encoded_input["attention_mask"])

    output = model(**encoded_input)
    
    print(i)
    print(output)  # [-2.9663,  2.3750]

    if output["logits"][0][0] < output["logits"][0][1]:
        answer_model.append(1)
    else:
        answer_model.append(0)


acc = accuracy_score(data_label, answer_model)
print("accuracy ",acc)   #only 0.74
`

 I have carefully checked the code for two days and retrained it, but the result is still different. I don't know where the problem might be. I hope to get your help！

Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.