I guess the difference is that in the tutorial they are initializing the current mini-batch’s hidden state as the previous mini-batch’s hidden state, which is why we need to detach them, otherwise we would unroll the rnn until the first data point as mentioned. If we are initializing each batch’s hidden state as zeros then we don’t need to do the detaching.