Recursive Backwards Q-Learning in Deterministic Environments
<|reference_start|>Recursive Backwards Q-Learning in Deterministic Environments: Reinforcement learning is a popular method of finding optimal solutions to complex problems. Algorithms like Q-learning excel at learning to solve stochastic problems without a model of their environment. However, they take longer to solve deterministic problems than is necessary. Q-learning can be improved to better solve deterministic problems by introducing such a model-based approach. This paper introduces the recursive backwards Q-learning (RBQL) agent, which explores and builds a model of the environment. After reaching a terminal state, it recursively propagates its value backwards through this model. This lets each state be evaluated to its optimal value without a lengthy learning process. In the example of finding the shortest path through a maze, this agent greatly outperforms a regular Q-learning agent.<|reference_end|>
@article{diekhoff2024recursive,
    title={Recursive Backwards Q-Learning in Deterministic Environments},
    author={Jan Diekhoff, J"orn Fischer},
    journal={arXiv preprint arXiv:2404.15822},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15822},
    primaryClass={cs.AI cs.LG}
}