Hi Prerna, thanks for answering!
But that wasn’t what I was asking… What I don’t get is what happens in a single run.
The way I understand it is this:
input is a tensor of size (num_cells, num_batches, len_inputs) i.e. an input vector for every cell of every batch.
hidden is a tensor of size (num_layers, num_batches, len_hidden) i.e. every batch in the run gets a hidden initial for every layer.
Is that correct?
I don’t want it to give it the initial hidden for every batch, but would like it to use the hidden from the previous batch.
I’d like it to do this without the loop:
len_inputs = ...
num_layers = ...
len_hidden = ...
h = (torch.zeros(num_layers, 1, len_hidden))
c = copy.deepcopy(h)

for i in range(len(subsequences)):  
    optimizer.zero_grad()
    out, h = model(subsequences[i], h)   
    loss = nn.WhateverLoss(out, subsequences[i])
    optimizer.step()

I don’t understand if that’s what happens under the hood when I tell it:
h = (torch.zeros(num_layers, num_subsequences, len_hidden))
c = copy.deepcopy(h)
out, h = model(subsequences)

And if not, how do I make it so?
Hope I made it clear, please tell me if not I’m having trouble with this for a while…
Thanks!