Hello, I have a similar problem. I used metric_for_best_model = eval_f1 for the model. But EarlyStopping stops the model even when f1 score is increasing. I did not include greater_is_better in my training arguments. Should I include it or not?
training_args = TrainingArguments(
f'./clickbait_identification/{dir_path}',

evaluation_strategy = "steps",

eval_steps=1000,

save_strategy = 'steps',

save_steps =1000,

learning_rate=2e-5,

per_device_train_batch_size=batch_size,

per_device_eval_batch_size=batch_size,

num_train_epochs=20,

weight_decay=0.01,

lr_scheduler_type='linear',

load_best_model_at_end=True,

metric_for_best_model='eval_f1',

logging_strategy ='epoch',

group_by_length=True,

seed=42

)

image694×293 11.5 KB