Hi all,
I was reading through the encoder decoder 23 transformers and saw how loss was generated. But I’m just wondering how it is internally generated?
Is it something like the following: Suppose I have the following pair: ("How are you?", "I am doing great"). In this case, is it calculating the cross entropy loss for the four output tokens and then averaging them?