You are currently flattening the input tensors such that the channel dimension would move into the batch dimension:
images = images.view(-1, seq_len, input_size)

and I guess this could yield the shape mismatch for an uneven batch size (the last one in the epoch).
Also, make sure this is indeed the use case you would like to apply, as it’s usually unwanted to move treat the different color channels of an input image as separate samples in the batch.