Hi John,
I am new to pytorch. I ran into similar troubles while attemting to train an lstm based language model. Have you been able to solve this? Might I suggest using 3 separate linear projection layers to project each of your targets into 3 different spaces before taking a softmax and calculating error?