You could use the last time step of your RNN and feed only this tensor to the linear layer or alternatively try to reduce the temporal dimension (e.g. via a mean() calculation etc.).
I’m not sure which approach would work better, so you might need to try out different approaches.