Reinforcement Learning, Monte Carlo Method, Machine Learning, Artificial Intelligence.
call the first time it is visited in an episode the first visit to s. The first-visit MC method estimates Vπ(s) as the average of the returns following first visits to s, whereas the every-visit MC method averages the returns following all visits to s. The mean return of a state where 1[St=s] is a