training_epoch_end just fetches some data from the output (such as a few losses) and creates metrics for logging. check_and_save finally saves this data in a checkpoint (one per epoch), after which the data is no longer needed and the next epoch can be begin.
I removed the training_epoch_end line, as well as check_and_save and the problems are still persisting.