Solved by tom in post #9 
              
              
                So if you run anomaly mode, the error might be thrown between the producing of NaN and before the hook is applied. (I must admit I don’t know and in lieu of a simple snippet I can copy-paste into PyTorch, I’m not going to start experiments, sorry.) 
One way around this might be to define your own au…