There’ve been other questions on this forum asking about LogSoftmax vs Softmax. This question is more focused on why LogSoftmax is claimed to be better (both numerically and in terms of speed) than applying Log to the output of Softmax. The claim is mentioned in this doc page:
https://pytorch.org/docs/stable/generated/torch.nn.functional.log_softmax.html 1
But, softmax by itself is actually numerically stable, and also uses the max trick for numerical stability (see link below). Therefore, it’s unclear to me why adding log to it makes it unstable.

  

      stackoverflow.com
  

  
      
    
  


  Numercially stable softmax 1



  python, numpy, nan, scientific-computing



  asked by
  
  
    Abhishek Bhatia
  
  on 06:11PM - 04 Mar 17 UTC


  

  
    
    
  

  


I’d really appreciate if someone can help me understand this via some math / code.