Hello Mohsen!



 moh_sen:




 KFrank:

Let input be a tensor of shape (nBatch, nClass) .


The input is not shape of ‘(n batch, nClass)’. The input is in shape (‘nBatch, mLogit’). i.e., each row has 1000 elements, corresponding to each classes.
I want to get sum over all classes.


Whether we call this number, 1000, nClass or mLogit is immaterial.
I maintain that my expression reproduces the result of your (corrected)
pseudo-code.  Have you tried it?
If you come to the contrary conclusion, please post a complete,
runnable sample script that illustrates your contrary result.  (You
can generate random sample data for you input tensor, but please
trim down the number 1000 to something more manageable like
5 or 10 to make everyone’s life easier.)
Also, to get our vocabulary straight, what do you mean by “class”
when you say “sum over all classes?”  How many classes do you
have in your example?  Should I understand the value of the quantity
“C” in the image of the equation you posted to be your number of
classes (and equal to 1000 in your example)?
Best.
K. Frank