.
reward we would receive after taking action a . Similarly, the transition function lets us see the successor state we would transition to after taking action a. A 1-step lookahead simply uses Bellman’s Equation as follows: Notation q^​(s, a) is the action-value function ′s′ is the successor state —