Hi Ayman,
If you don’t feed back the hidden state of t to the step t+1, you basically use a fancy version of a linear layer with shared weights for all time steps, but you loose the recurrent property of the model (I think this approach is called time distributed layer, which is not a recurrent one). Whenever you have a time series the states of a sequence are highly correlated so I don’t see a problem with that (since that is an inherent property to all series, there is nothing specific about RL in this case). Or am I getting your question wrong?
Also, you always have to feed the output to the next layer not the hidden state.