The cell implementations take one timestep at a time. The LSTM, RNN and GRU all take inputs with several timesteps in one go. I find it helpful to be very clear about the distinction between the batch dimension, whose indices correspond to different input sequences, and the sequence dimension whose indices correspond to different timesteps of each sequence. The term “batch” can be a little ambiguous if you are not careful.
I believe that the GPU implementation of LSTM is somewhat faster that the cell implementation, but I have never tried it myself.
I think most people either use LSTMs for short sequences (e.g. sentences to be classified) and they pass entire sequences to the LSTM in one go, or for long sequences which they cut up into subsequences.
I have found that if I take a really long sequence and divide it up into subsequences which I stack into batches, then I can get through an epoch of training much faster. That said, I didn’t start out by doing that, I started with simpler methods and I went from there.