MLP-KAN: Unifying Deep Representation and Function Learning
Recent advancements in both representation learning and function learning
have demonstrated substantial promise across diverse domains of artificial
intelligence. However, the effective integration of these paradigms poses a
significant challenge, particularly in cases where users must manually decide
whether to apply a representation learning or function learning model based on
dataset characteristics. To address this issue, we introduce MLP-KAN, a unified
method designed to eliminate the need for manual model selection. By
integrating Multi-Layer Perceptrons (MLPs) for representation learning and
Kolmogorov-Arnold Networks (KANs) for function learning within a
Mixture-of-Experts (MoE) architecture, MLP-KAN dynamically adapts to the
specific characteristics of the task at hand, ensuring optimal performance.
Embedded within a transformer-based framework, our work achieves remarkable
results on four widely-used datasets across diverse domains. Extensive
experimental evaluation demonstrates its superior versatility, delivering
competitive performance across both deep representation and function learning
tasks. These findings highlight the potential of MLP-KAN to simplify the model
selection process, offering a comprehensive, adaptable solution across various
domains. Our code and weights are available at
https://github.com/DLYuanGod/MLP-KAN.
6704617783f1ca2d2dccb67e
Yunhong He
admin_assigned
2024-10-08T07:25:10.837Z
66ac9621c40d4b95e7d4417f
/avatars/b2f2fe6cd185a12483217ff099daca06.svg
Yunhong He
user
yunhong1
6704617783f1ca2d2dccb67f
Yifeng Xie
admin_assigned
2024-10-08T07:25:16.291Z
63ce7b5781065f5e3fff8c30
/avatars/a394f61554bd21b0545a240821e25347.svg
Yifeng Xie
user
Evfidiw
6704617783f1ca2d2dccb680
Zhengqing Yuan
admin_assigned
2024-10-08T07:25:27.350Z
62f3cc40b3e6016d46027e1d
/avatars/9b26d2fd5e8574f633ffd97af1700b45.svg
YuanZhengqing
user
YuanZhengqing
6704617783f1ca2d2dccb681
Lichao Sun
admin_assigned
2024-10-08T07:25:33.102Z
65a52766215aabac489e3468
/avatars/fe05e22cd7e12e961296426434e17c76.svg
Lichao Sun
user
sunlichao137
MLP-KAN: Unifying Deep Representation and Function Learning
Recent advancements in both representation learning and function learning
have demonstrated substantial promise across diverse domains of artificial
intelligence. However, the effective integration of these paradigms poses a
significant challenge, particularly in cases where users must manually decide
whether to apply a representation learning or function learning model based on
dataset characteristics. To address this issue, we introduce MLP-KAN, a unified
method designed to eliminate the need for manual model selection. By
integrating Multi-Layer Perceptrons (MLPs) for representation learning and
Kolmogorov-Arnold Networks (KANs) for function learning within a
Mixture-of-Experts (MoE) architecture, MLP-KAN dynamically adapts to the
specific characteristics of the task at hand, ensuring optimal performance.
Embedded within a transformer-based framework, our work achieves remarkable
results on four widely-used datasets across diverse domains. Extensive
experimental evaluation demonstrates its superior versatility, delivering
competitive performance across both deep representation and function learning
tasks. These findings highlight the potential of MLP-KAN to simplify the model
selection process, offering a comprehensive, adaptable solution across various
domains. Our code and weights are available at
https://github.com/DLYuanGod/MLP-KAN.
MLP-KAN integrates MLPs and KANs in a MoE architecture within a transformer framework to adaptively handle both representation and function learning tasks, achieving competitive results across diverse datasets.
Multi-Layer Perceptrons MLPs Kolmogorov-Arnold Networks KANs Mixture-of-Experts MoE transformer-based framework