Still not fixed actually. Check this repo, the categorical logits are nan’ing out:
  
      
      Pastebin
  
  
    

import torchfrom matplotlib import pyplot as pltimport numpy as npNSAM -... 1

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.


  
  
    
    
  
  


Okay, I fixed it by applying a tanh before the final linear layer. No more nans, the tanh chokes the magnitude of the final layer…