You can keep a track of F1 on the validation data and decide when to stop the training based on the best F1. I am using ignite library to do a similar thing. You can create a metric: F1Weighted(Metric): using the Metric class from pytorch-ignite 9. Then, add the metric as follows:
evaluator = create_supervised_evaluator(model,
                                                metrics={'f_1': F1Weighted})

You can use earlystopping from ignite to stop training and also modelcheckpointing. Its all present in ignite.