I’m glad you’ve found this issue and the authors would be certainly happy, if you fix it. 
Note that even after calling model.eval(), the model will be trained (Autograd will still calculate gradients), but the behavior of some layers will change. E.g. batch norm layers will use the running stats instead of the batch stats and dropout will be disabled. This would certainly explain a poor training performance.