Reinforcement Learning, Monte Carlo Method, Machine Learning, Artificial Intelligence.
binary indicator function. We may count the visit of state s every time so that there could exist multiple visits of one state in one episode (“every-visit”), or only count it the first time we encounter a state in one episode (“first-visit”). Similarly, the approximation could easily be applied to