Solved by ptrblck in post #5 
              
              
                nn.CrossEntropyLoss for a mutli-class classification on a temporal sequence expects the model output in the shape [batch_size, nb_classes, seq_len] containing logits, while the target should be passed as [batch_size, seq_len] containing the class indices in the range [0, nb_classes-1].