It is an implementation fault that I have written the code that way, but in this case mini_batch_size*sequence_length is the same as 24*20000.
In other words MINI_BATCH_SIZE == 24
SEQUENCE_LENGTH == 20000
The reason why I’m reshaping is for CrossEntropyLoss.
The model works well in general, but sometimes spits that high values.