  Counterfactual Risk Minimization (CRM) is a framework for dealing with the
logged bandit feedback problem, where the goal is to improve a logging policy
using offline data. In this paper, we explore the case where it is possible to
deploy learned policies multiple times and acquire new data. We extend the CRM
principle and its theory to this scenario, which we call "Sequential
Counterfactual Risk Minimization (SCRM)." We introduce a novel counterfactual
estimator and identify conditions that can improve the performance of CRM in
terms of excess risk and regret rates, by using an analysis similar to restart
strategies in accelerated optimization methods. We also provide an empirical
evaluation of our method in both discrete and continuous action settings, and
demonstrate the benefits of multiple deployments of CRM.

Machine Learning