Model based learning


- Learn an approximate model based on experiences 
- Solve for values as if the learned model were correct?
![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F8a46ac1d-3c48-4aa1-a780-fbe3036338a1%2FUntitled.png?table=block&id=92448420-b90c-4be0-81c7-8a7c47bda2f8&cache=v2)
> Model free learning do not instersted in R and T



**Step 1: Learn empirical MDP model**
- Count outcomes s’ for each s, a
- Normalize to give an estimate of `math: T\hat(s, a, s')` - 단순확률평균 of action
**Step 2: Solve the learned MDP**
use value iteration, as before
> State value function - sum of rewards get from states passed (all time)
> Action value function - expected reward from action in this state (that time)





