Reinforcement Learning, Monte Carlo Method, Machine Learning, Artificial Intelligence.
states. An obvious way to estimate it is simply averaging the returns observed after visits to that state. These observed visits are from the experience that is recorded by interaction with the environment. As more returns are observed, the closer the estimates will converge to the expected value.