Hi, I investigated the code with debugger,
and I checked whether there is labels before I put my eval_dataset (in case of evaluation) to trainer.evaluate(). code example
I got batched eval_dataset with shape (batch_size, 6) which is consist of
['attention_mask', 'input_ids', 'label', 'sentence1', 'sentence2', 'token_type_ids'], and there were proper labels as you concerned.
Is there any ways to get access inside of the inner method evaluation_loop so I can check how it works?