Solved by Shreyansh_IITB in post #13 
              
              
                Thanks @ptrblck for the help, I finally found the issue. 
[image] 
These alpha here denotes probability after projection over target vocabulary, I was implementing this equation as is. 
As you can see there is a summation over multiplication of probabilities due to this alpha values were underflowin…