Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization
<|reference_start|>Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization: This work studies a Reinforcement Learning (RL) problem in which we are given a set of trajectories collected with K baseline policies. Each of these policies can be quite suboptimal in isolation, and have strong performance in complementary parts of the state space. The goal is to learn a policy which performs as well as the best combination of baselines on the entire state space. We propose a simple imitation learning based algorithm, show a sample complexity bound on its accuracy and prove that the the algorithm is minimax optimal by showing a matching lower bound. Further, we apply the algorithm in the setting of machine learning guided compiler optimization to learn policies for inlining programs with the objective of creating a small binary. We demonstrate that we can learn a policy that outperforms an initial policy learned via standard RL through a few iterations of our approach.<|reference_end|>
@article{marinov2024offline,
    title={Offline Imitation Learning from Multiple Baselines with Applications to
  Compiler Optimization},
    author={Teodor V. Marinov, Alekh Agarwal, Mircea Trofin},
    journal={arXiv preprint arXiv:2403.19462},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.19462},
    primaryClass={cs.LG cs.PL}
}