Reinforcement Learning, Machine Learning, Monte Carlo, Data Science, Thoughts And Theory.
approach of Q-function estimation: the high number of total pairs have (state, action) many of which might never be visited. If the policy is deterministic, then for a given state, it might always greedily choose only one action in the following learning iterations. As a result, we will never be