Not sure about the definition of RNNCell in Pytorch, but quoting from TF Tutorial on RNN 10:

In addition to the built-in RNN layers, the RNN API also provides cell-level APIs. Unlike RNN layers, which processes whole batches of input sequences, the RNN cell only processes a single timestep.