Sketch-and-Solve: Optimized Overdetermined Least-Squares Using Randomized Numerical Linear Algebra
<|reference_start|>Sketch-and-Solve: Optimized Overdetermined Least-Squares Using Randomized Numerical Linear Algebra: Sketch-and-solve is a powerful paradigm for tackling large-scale computational problems by reducing their dimensionality using sketching matrices. This paper focuses on applying sketch-and-solve algorithms to efficiently solve the overdetermined least squares problem, which is fundamental in various domains such as machine learning, signal processing, and numerical optimization. We provide a comprehensive overview of the sketch-and-solve paradigm and analyze different sketching operators, including dense and sparse variants. We introduce the Sketch-and-Apply (SAA-SAS) algorithm, which leverages randomized numerical linear algebra techniques to compute approximate solutions efficiently. Through extensive experiments on large-scale least squares problems, we demonstrate that our proposed approach significantly outperforms the traditional Least-Squares QR (LSQR) algorithm in terms of runtime while maintaining comparable accuracy. Our results highlight the potential of sketch-and-solve techniques in efficiently handling large-scale numerical linear algebra problems.<|reference_end|>
@article{lavaee2024sketch-and-solve:,
    title={Sketch-and-Solve: Optimized Overdetermined Least-Squares Using
  Randomized Numerical Linear Algebra},
    author={Alex Lavaee},
    journal={arXiv preprint arXiv:2409.14309},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.14309},
    primaryClass={cs.LG cs.NA math.NA}
}