Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
<|reference_start|>Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting: In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information.<|reference_end|>
@article{ilbert2024analysing,
    title={Analysing Multi-Task Regression via Random Matrix Theory with
  Application to Time Series Forecasting},
    author={Romain Ilbert, Malik Tiomoko, Cosme Louart, Ambroise Odonnat, Vasilii
  Feofanov, Themis Palpanas, Ievgen Redko},
    journal={arXiv preprint arXiv:2406.10327},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10327},
    primaryClass={stat.ML cs.LG}
}