Machine Learning.
be decomposed into immediate reward and discounted value of successor state 𝛾V(St+1) v(s)=E[Gt |St =s] =E[Rt+1+γRt+2+γ2Rt+3+…|St =s] =E[Rt+1+γ(Rt+2+γRt+3+…) |St =s] =E[Rt+1+γGt+1 |St =s] =E[Rt+1+γv(St+1)|St =s] Bellman equation can be put in a matrix form -> v = R + γPv Bellman equation is linear