A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix Decomposition
<|reference_start|>A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix Decomposition: Recently, there has been a growing interest in the exploration of Nonlinear Matrix Decomposition (NMD) due to its close ties with neural networks. NMD aims to find a low-rank matrix from a sparse nonnegative matrix with a per-element nonlinear function. A typical choice is the Rectified Linear Unit (ReLU) activation function. To address over-fitting in the existing ReLU-based NMD model (ReLU-NMD), we propose a Tikhonov regularized ReLU-NMD model, referred to as ReLU-NMD-T. Subsequently, we introduce a momentum accelerated algorithm for handling the ReLU-NMD-T model. A distinctive feature, setting our work apart from most existing studies, is the incorporation of both positive and negative momentum parameters in our algorithm. Our numerical experiments on real-world datasets show the effectiveness of the proposed model and algorithm. Moreover, the code is available at https://github.com/nothing2wang/NMD-TM.<|reference_end|>
@article{wang2024a,
    title={A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix
  Decomposition},
    author={Qingsong Wang, Chunfeng Cui, Deren Han},
    journal={arXiv preprint arXiv:2402.02442},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.02442},
    primaryClass={cs.LG eess.IV}
}