The training loop repackages the hidden state before each batch of training data which means that bptt (back propagation through time) is cut off at the beginning of each batch.
So if you want training to be able to back propagate through 50 timesteps then each batch must contain at least 50 timesteps.