The closest thing might be I use a backward hook to log grads during training,not sure if that matters.

Can you share the code of the hook? If the hook return something bad, it could cause the error above.