Solved by ptrblck in post #8 
              
              
                The values of squared_norm in PrimaryCaps explode and create the NaNs. 
In the last iteration before the NaNs are raised PrimaryCaps creates tensors with these statistics: 
print(squared_norm.min(), squared_norm.max())
> tensor(1.4527e+20, device='cuda:0', grad_fn=<MinBackward1>) tensor(5.3600e+26, …