Glad you’re trying the native API, and thanks for the minimal repro.  I think this indicates my wrappers aren’t reflattening weights in the way cudnn rnns require.  Cudnn rnn support is a high priority, I’ll look at it soon.  In the meantime, the cell-based RNN API should work.