As a guess, evaluation requires a generation procedure whereas training uses teacher-forcing and cross-entropy loss. This might not be the case though because the validation step may include some generation - I would highly recommend using the Pycharm debugger to test out what part of the code is taking so long.