Well u can still use an LSTM with some dense layers as u initially did
There’s really no rule of thumb here or sth like that.
Hmmmm🤔
I’m kinda curious tho. How would it affect the decision it takes next?
Coz from my knowledge given a state the action selected is the one with the maximum approximate value of Q outputted by the network
Or is there sth else u wish to do with this?