Usually you would calculate the validation error/loss and save the best performing model (i.e. with the highest validation accuracy).
Have a look at the ImageNet example 1.2k to see, how save_checkpoint is used for the best accuracy.