Thank you so much for your answer, it seems I wrote my reply at same time as you (what a coincidence).
So take-away here is that seq length doesn’t actually matter and basically since we can’t sometimes take huge sequence at once in memory we have this seq length parameter?
I’m sorry it’s just bit hard for me to understand this concept of LSTM properly, don’t mind me. So what I think right now is that is the hidden state, cell state after end of 1 sequence is fed as initial hidden state and cell state for sequence 2 in same batch??
Weight updates happen in batches right? I’m struggling to visualize how the long term dependency is maintained when we use batches.
You said I shouldn’t pass states across iterations but I hoped that my data point r (row number r) can maybe make better prediction if model has seen prediction for all rows before r. Hence I currently am passing hidden state and cell state across each forward pass (ie across batches right)
Could you shed some clarity on this please