This code is unfortunately not executable.
Anyway, it looks like you are setting the model(s) to .eval() after loading.
Is it also the case for the continuation of your training?
If so, you should set them to .train() mode.