Thanks for your reply @ptrblck.
Yes, I am using nn.BCELoss with a softmax output from the decoder.
data represents a one hot encoded vector of shape [600, 120, 33] (120 is the length of each string and 33 is the length of the character set used to make these strings).
I tried training the model on a small data sample of 10 samples. The initial loss was 20.636. The loss stagnated at 19.803 after around 7500 epochs. The loss decreased very slowly but continuously through these epochs.
So, I guess it is not overfitting at all.
Please let me know if I can clarify anything else about the code.
Any ideas on what can be done @ptrblck ? Thanks.