Solved by ptrblck in post #2 
              
              
                Try to permute the dimensions in your predication tensor to [batch_size, nb_classes, seq_len], i.e. [2, 5, 3] and it should work.