I will try to clarify what I think you are asking as best as I can .



 mfluegge:

If I pick batch size = 4, that means I have created 4 subsequences and 2 batches


1 - yes



 mfluegge:

This number of batches is the seq_len dimension in the RNN input


2- No. The seq_len parameter is the size of your batch, the number of batches corresponds to the batch dimension in the LSTM input.



 mfluegge:

I guess I can now decide how many batches I should use per training iteration; the number of batches I input into my network per training iteration determines how long the training sequence used for an update is


3- Yes. You can decide up to how many batches you want to backpropagate. If you have only 4 batches and you want to backpropagate through all of them until the beginning of your sequence then I’m guessing you can call loss.backward(retain_graph=True) at the end of every batch. I don’t think this is very common since when having lots of data it will be very slow.



 mfluegge:

I guess I should reset the hidden states after inputting all the batches once and thereby finishing an epoch


4 -When to reset the hidden state depends on what you want. You should reset your hidden state when you want your RNN to not have any notion at all of what happened earlier. If you think its important to remember all of the data then you reset the hidden state after a full epoch but if one batch is independent from the rest then you can reset it after each .backward(). My questions and @jpeg729 answers on this same thread could clarify or expand on this.
That said, I am a bit of a newbie so I hope someone with more knowledge could confirm or correct my answer.  Especially points 3 & 4.
Lastly, never hesitate to ask these kind of questions. This is the very purpose of these forums and furthermore they help newbies like me to understand things better.