From my point of view, the batch_size and sequence_step used in this demo are 1 (i.e., mini_batch) and the length of a sentence, respectively.
In other words, a sentence is a batch (because loss is calculated after each batch) and the loss is propagating back through the whole input of a sentence.
I hope I got the right logic since I’m also a noob playing around with PyTorch.