It would still be an RNN in the forward pass, but the gradients wouldn’t propagate recurrently.
A colleague suggested a clever way to do this (using RNN, not RNNCell):

Forward pass with the RNN as usual and get the sequence of hidden states hidden (of size b x l x h)
Transform hidden to be size bl x h and detach; repeat the input l times to create a new batch of size bl so your input has size bl x 1 x X and hidden is of size bl x 1 x h (basically, sequences of length 1)
Call the same RNN again with these two tensors as input instead, and take the output of this call as the final output which will be used to compute the gradients.

It does require two forward passes instead of one, but is still taking advantage of the efficiency of the RNN forward pass.