Hi @ptrblck. Thank you for the prompt response. My validation dataset is the same as testset currently as  I am not tuning any hyperparameters using it. Also, I don’t understand what you mean by the package with the trainer class. I am not using anything different than what is normally used if that helps. Also, the Trainer class is a bit messy and hard to understand. Hence, haven’t posted it here.