Reinforcement Learning, Monte Carlo Method, Machine Learning.
second episode visits state s1​ at the second last step, and the third episode visits state s1​ at the first step, and so on. Our goal is to calculate the average returns of each state after visits to that state. Depending on the way of calculating averaged returns, we have First-Visit MC and