I am systematically encountering a strange behavior at the beginning of the predicted sequence when using torch.nn.RNN. This phenomena is not present when using torch.nn.RNNCell.

RNNvsRNNCell.png943×369 43.9 KB

The code to reproduce this behavior can be found in: https://github.com/landajuela/PyTorch-examples/blob/master/Signal2Signal.ipynb 12 by switching cell_model = True or False in the training cell.
It seems an initialization problem, but I am not able to correct it.