Reinforcement Learning, Machine Learning, Monte Carlo, Data Science, Thoughts And Theory.
distribution from the interaction of the agent with the environment might differ a lot from the one used for exploring starts. As a consequence, the agent might learn the optimal policy based on the unreal data distribution. Therefore, the learned policy will not be optimal in the real environment.