Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift
Zhen Zhang Xin Liu Shaoli Wang Jiaye Teng
Covariate shift occurs when the distribution of input features differs between the training and testing phases.  In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its common occurrence in real-world scenarios. In this paper, we investigate the minimax lower bound of the problem when the source and target distributions are known. To achieve the minimax optimal bound (up to a logarithmic factor), we propose a two-stage algorithm. Specifically, it first trains an optimal estimator for the function under the source distribution, and then uses a likelihood ratio reweighting procedure to calibrate the moment estimator. In practice, the source and target distributions are typically unknown, and estimating the likelihood ratio may be unstable. To solve this problem, we propose a truncated version of the estimator that ensures double robustness and provide the corresponding upper bound. Extensive numerical studies on synthetic examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method.
covariate shift, minimax optimal, two-stage algorithm, double robustness
This paper investigates the minimax lower bound for moment estimation under covariate shift, and proposes a minimax optimal two-stage algorithm with a truncated version that ensures double robustness.