I’m a bit confused. Why do you want the network to output same hidden state for each data in the batch? If so, there will be no correlation along the time dimension, and you would be essentially predicting current timestep output from current timestep input only. If that is something you want in your use case, maybe try non-recurrent models?