oh ok I see how that can work though doesn’t seem like it would be very robust in general purpose algorithms go. As for discount factor in my understanding and use of it a lower discount factor will just lead to grab rewards as soon as it can(High immediate reward values) and a higher discount factor leads to more importance on later rewards. Gridworld is a end goal objective game I believe(sorry never played lol) so would benefit with higher D factor in my opinion. Number of steps should not be an issue.