Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
<|reference_start|>Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm: Given a dataset of expert demonstrations, inverse reinforcement learning (IRL) aims to recover a reward for which the expert is optimal. This work proposes a model-free algorithm to solve entropy-regularized IRL problem. In particular, we employ a stochastic gradient descent update for the reward and a stochastic soft policy iteration update for the policy. Assuming access to a generative model, we prove that our algorithm is guaranteed to recover a reward for which the expert is $\varepsilon$-optimal using $\mathcal{O}(1/\varepsilon^{2})$ samples of the Markov decision process (MDP). Furthermore, with $\mathcal{O}(1/\varepsilon^{4})$ samples we prove that the optimal policy corresponding to the recovered reward is $\varepsilon$-close to the expert policy in total variation distance.<|reference_end|>
@article{renard2024convergence,
    title={Convergence of a model-free entropy-regularized inverse reinforcement
  learning algorithm},
    author={Titouan Renard, Andreas Schlaginhaufen, Tingting Ni, Maryam Kamgarpour},
    journal={arXiv preprint arXiv:2403.16829},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.16829},
    primaryClass={cs.LG cs.AI}
}