@ZeweiChu because if hidden states are None , they are initialized as zero tensors, then they are used for computation of next hidden states and RNN outputs new hidden states. You can look at source code and check it