The training state is also a part of state_dict and will thus be restored when you load you trained model.
After load_state_dict(), you should call dual_encoder.eval() or dual_encoder.training == True. (IIRC, they do the same thing)