Thank you for your answer.
But here I don’t really understand.
First about the input : My first layer is

nn.Linear(input_size, hidden_size)

so what is the input size here? Normally in Keras i just put input_size = (batch, seq_len, no_classes) but for Pytorch I have tried and the only satisfied value for input_size is 20 which is my no_classes? So normally with Pytorch we should pout as no_layers for input size?
Second one, I understand the permute() you so me so to fit with the loss function, I think I have to make the target_size as [batch_size, seq_len] so that mean I will not need to convert my level to one-hot encoding as Keras I believe. But can you so me how to determine the number of sample for each batch? With Keras I just declare the batch_size = no_samples per batch only.
Sorry for the newbie question, I’m really new with Pytorch , just move from Keras to this framework so my idea just cannot follow Pytorch properly and tutorial on the internet is not as detail as Keras ( I think because Pytorch is still new ).