Distributed Source Coding for Parametric and Non-Parametric Regression
<|reference_start|>Distributed Source Coding for Parametric and Non-Parametric Regression: The design of communication systems dedicated to machine learning tasks is one key aspect of goal-oriented communications. In this framework, this article investigates the interplay between data reconstruction and learning from the same compressed observations, particularly focusing on the regression problem. We establish achievable rate-generalization error regions for both parametric and non-parametric regression, where the generalization error measures the regression performance on previously unseen data. The analysis covers both asymptotic and finite block-length regimes, providing fundamental results and practical insights for the design of coding schemes dedicated to regression. The asymptotic analysis relies on conventional Wyner-Ziv coding schemes which we extend to study the convergence of the generalization error. The finite-length analysis uses the notions of information density and dispersion with additional term for the generalization error. We further investigate the trade-off between reconstruction and regression in both asymptotic and non-asymptotic regimes. Contrary to the existing literature which focused on other learning tasks, our results state that in the case of regression, there is no trade-off between data reconstruction and regression in the asymptotic regime. We also observe the same absence of trade-off for the considered achievable scheme in the finite-length regime, by analyzing correlation between distortion and generalization error.<|reference_end|>
@article{wei2024distributed,
    title={Distributed Source Coding for Parametric and Non-Parametric Regression},
    author={Jiahui Wei, Elsa Dupraz and Philippe Mary},
    journal={arXiv preprint arXiv:2404.18688},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.18688},
    primaryClass={cs.IT math.IT}
}