I’m still not quite sure I understand. The hidden state for each batch which goes through forward pass is always zeros correct? So what difference would it make in terms of detaching the initial hidden state, from using a new hidden state for each new batch???
Also in this 18 example in pyTorch Docs, init_hidden is called for each input sequence. Aren’t they resetting hidden state to zero in this case(which btw also detaches the history)?