Probabilistic Examination of Least Squares Error in Low-bitwidth Cholesky Decomposition
<|reference_start|>Probabilistic Examination of Least Squares Error in Low-bitwidth Cholesky Decomposition: In this paper, we propose a new approach to justify a round-off error impact on the accuracy of the linear least squares (LS) solution using Cholesky decomposition. This decomposition is widely employed to inverse a matrix in the linear detector of the Multi-User multi-antenna receiver. The proposed stochastic bound is much closer to actual errors than other numerical bounds. It was tested with a half-precision format and validated in realistic scenarios. Experimental results demonstrate our approach predicts errors very close to those achieved by simulations. The proposed approach can be employed to analyze the resulting round-off error in many other applications.<|reference_end|>
@article{osinsky2024probabilistic,
    title={Probabilistic Examination of Least Squares Error in Low-bitwidth
  Cholesky Decomposition},
    author={Alexander Osinsky, Roman Bychkov, Mikhail Trefilov, Vladimir Lyashev,
  Andrey Ivanov},
    journal={arXiv preprint arXiv:2404.05082},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.05082},
    primaryClass={cs.IT math.IT}
}