MLP-KAN: Unifying Deep Representation and Function Learning
<|reference_start|>MLP-KAN: Unifying Deep Representation and Function Learning: Recent advancements in both representation learning and function learning have demonstrated substantial promise across diverse domains of artificial intelligence. However, the effective integration of these paradigms poses a significant challenge, particularly in cases where users must manually decide whether to apply a representation learning or function learning model based on dataset characteristics. To address this issue, we introduce MLP-KAN, a unified method designed to eliminate the need for manual model selection. By integrating Multi-Layer Perceptrons (MLPs) for representation learning and Kolmogorov-Arnold Networks (KANs) for function learning within a Mixture-of-Experts (MoE) architecture, MLP-KAN dynamically adapts to the specific characteristics of the task at hand, ensuring optimal performance. Embedded within a transformer-based framework, our work achieves remarkable results on four widely-used datasets across diverse domains. Extensive experimental evaluation demonstrates its superior versatility, delivering competitive performance across both deep representation and function learning tasks. These findings highlight the potential of MLP-KAN to simplify the model selection process, offering a comprehensive, adaptable solution across various domains. Our code and weights are available at \url{https://github.com/DLYuanGod/MLP-KAN}.<|reference_end|>
@article{he2024mlp-kan:,
    title={MLP-KAN: Unifying Deep Representation and Function Learning},
    author={Yunhong He, Yifeng Xie, Zhengqing Yuan, Lichao Sun},
    journal={arXiv preprint arXiv:2410.03027},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.03027},
    primaryClass={cs.LG cs.CL}
}