lubiluk:

Is it a good idea to always wrap model calls with eval/train?


Yes, I would recommend to always call model.train() before the training and model.eval() before the evaluation or testing of the model. Even if your current model does not use any batchnorm or dropout layers, the model itself or custom layers might use the self.training attribute to change their behavior.