CUDNN only supports a handful of well-known RNN cell types. If you want to implement your own RNN cell, modifying or subclassing RNNBase doesn’t really work. Instead, you should subclass RNNCellBase and write something that looks like nn.RNNCell (https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/rnn.py#L353 56). Then use a for-loop to unroll the RNN cell manually.