.
(s, a). The action-value function is usually denoted qπ​(s, a). The action-value qπ​(s,a) for state-action pair (s,a) is the value associated with taking action a from state s and then following policy π . For now, we’ll calculate action values using a “1-step lookahead”. Instead of calculating the