Hey Sven,
So the way I see it is that we can either do one of two things:

Collapse a dimension into another dimension
Some sort of operation to reduce a dimension

A straight forward approach would be to do something like this: x = x[batch_size, -1, height, width]. This would collapse the timestep information into the channels. This might be problematic because future operations won’t strictly know about the temporal aspect, but there’s still some ordering to it all (timestep-1 features, timestep-2 features, etc.).
I do like the idea of a 3D convolution, I think it can do a good job of embedding your temporal dimension. You may also consider swapping the time dimension and num_channels dimension before the convolution. You can stride over time and reduce it that way (while also increasing your feature map). I would suggest the latter, I think it can do a better job with embedding the temporal information.