But after the first iteration/step of training should the first hidden layer still be all zeroes. I’m guessing not since when we’re training the first hidden layer weight values change. In that case:
On my first training step:
my h0 has changed from all zeroes to some non-zero weights - let’s say h0*.
On my second training step:
how do I specify that I want the updated h0*
i.e. lstm_out, _ = self.lstm(x.view(-1, 1, 3),h0*)
without sharing the hidden layers from one invocation the next.
I’m sure I’m confusing something here - but I just wanted to double check.