Hello,
I am having problems with the EarlyStoppingCallback I set up in my trainer class as below:
training_args = TrainingArguments(
    output_dir = 'BERT',
    num_train_epochs = epochs,
    do_train = True,
    do_eval = True,
    evaluation_strategy = 'epoch',
    logging_strategy = 'epoch',
    per_device_train_batch_size = batch_size,
    per_device_eval_batch_size = batch_size,
    warmup_steps = 250,
    weight_decay = 0.01,
    fp16 = True,
    metric_for_best_model = 'eval_loss',
    load_best_model_at_end = True
)

trainer = MyTrainer(
    model = bert,
    args = training_args,
    train_dataset = train_dataset,
    eval_dataset = val_dataset,
    compute_metrics = compute_metrics,
    callbacks = [EarlyStoppingCallback(early_stopping_patience = 3)]
)

trainer.train()

I keep getting the following error:

Screen Shot 2021-04-21 at 09.45.06954×561 78.7 KB

I already tried running the code without the metric_for_best_model arg, but it still gives me the same error.
I tweaked the Trainer class a bit to report metrics during training, and also created custom_metrics to report during validation. I suspect that maybe I made a mistake there and that’s why I can’t retrieve the validation loss now. See here 29 for the tweaked code.
Thanks in advance!!