Alright, I kinda feel bad asking all these stupid questions, but well, you asked for it 
I might have either kind of understood it now or gone fully mad.
Lets say I have one “big” overall sequence of steps 1,2,…,8
If I pick batch size = 4, that means I have created 4 subsequences and 2 batches
— b1 b2
s1 1, 2,
s2 3, 4,
s3 5, 6,
s4 7, 8
bn = batch n, sn = subsequence n
I guess I can now decide how many batches I should use per training iteration; the number of batches I input into my network per training iteration determines how long the training sequence used for an update is. This number of batches is the seq_len dimension in the RNN input. I guess the appropriate term for the sequence used in one training iteration would be “subsubsequence”? So if I for example chose to only use one batch per training iteration I would use a subsequence of each of the 4 subsequences s1-s4.
Assuming what I wrote until now is actually correct and not completely insane I would do updates using subsubsequences. I guess I should reset the hidden states after inputting all the batches once and thereby finishing an epoch. But I guess I should not reset the hidden states after every single update since I only updated based on a subsubsequence and it would make sense that the following subsubsequence would use its subsequence’s previous hidden state. Given that this is sensible I suppose I would have to save the hidden state at the end of every training iteration and use it to initialize the hidden state during the next training iteration.
Is this even close to correct? If not feel free to tell me where I’ve gone wrong. Thanks!