Checkpointing really becomes its own thing with HP search, and is completely untested. I would recommend not using anything fancy, and I would not expect load_best_model_at_end  to work during HP searches, especially with trials being aborted by the backend if they are unconclusive before we reach the end. load_best_model_at_end is really meant to be used in a regular training. That’s the same for 3, those arguments decide which model is the best and also what to monitor for early stopping, but they are not used by the hyperparameter search.