In that case you could also try to pass the spatial dimensions (height*width) as the temporal dimension and the channels as the feature dimension.
Still unsure, how well this would work, but might be a reasonable approach.