The typical thing is to do a reduction over the time axis, like max or mean.
fast.ai’s (with the reference being Howard and Ruder’s ULMFiT paper) advocate using max, mean andlast element for RNNs. As for CNNs the last element is hard / not that meaningful, you would concatenate max and mean.
Best regards
Thomas