Reinforcement Learning, Artificial Intelligence, Machine Learning, Data Science, Statistics.
Formal Justification in Model-Free Methods In model-free methods, formal justification for using the q-value instead of the value function lies in the inability to directly evaluate the q-function without the environment model. Without this model, substituting the q-function into the value function