title: KAN or MLP: A Fairer Comparison
summary: This paper does not introduce a novel method. Instead, it offers a fairer and
  more comprehensive comparison of KAN and MLP models across various tasks,
  including machine learning, computer vision, audio processing, natural language
  processing, and symbolic formula representation. Specifically, we control the
  number of parameters and FLOPs to compare the performance of KAN and MLP. Our
  main observation is that, except for symbolic formula representation tasks, MLP
  generally outperforms KAN. We also conduct ablation studies on KAN and find
  that its advantage in symbolic formula representation mainly stems from its
  B-spline activation function. When B-spline is applied to MLP, performance in
  symbolic formula representation significantly improves, surpassing or matching
  that of KAN. However, in other tasks where MLP already excels over KAN,
  B-spline does not substantially enhance MLP's performance. Furthermore, we find
  that KAN's forgetting issue is more severe than that of MLP in a standard
  class-incremental continual learning setting, which differs from the findings
  reported in the KAN paper. We hope these results provide insights for future
  research on KAN and other MLP alternatives. Project link:
  https://github.com/yu-rp/KANbeFair
paper_authors._id: 66a06d8c8a65778f7264be7f
paper_authors.name: Runpeng Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-24T07:25:24.009Z
paper_authors._id: 635364b3c41f548fe39db945
paper_authors.avatarUrl: /avatars/ad1916bbfabca0b6651c8eabacc5eba8.svg
paper_authors.fullname: Runpeng Yu
paper_authors.type: user
paper_authors.user: rp-yu
paper_authors._id: 66a06d8c8a65778f7264be80
paper_authors.name: Weihao Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T07:34:08.644Z
paper_authors._id: 5df833bdda6d0311fd3d5403
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5df833bdda6d0311fd3d5403/62OtGJEQXdOuhV9yCd4HS.png
paper_authors.fullname: Weihao Yu
paper_authors.type: user
paper_authors.user: whyu
paper_authors._id: 66a06d8c8a65778f7264be81
paper_authors.name: Xinchao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-24T08:01:30.175Z
paper_authors._id: 63fc03a50aab060792ffef39
paper_authors.avatarUrl: /avatars/9d5b1bb2a41928e08176b703935133ab.svg
paper_authors.fullname: Wangxinchao
paper_authors.type: user
paper_authors.user: wxcTest
paper_title: KAN or MLP: A Fairer Comparison
paper_summary: This paper does not introduce a novel method. Instead, it offers a fairer and
  more comprehensive comparison of KAN and MLP models across various tasks,
  including machine learning, computer vision, audio processing, natural language
  processing, and symbolic formula representation. Specifically, we control the
  number of parameters and FLOPs to compare the performance of KAN and MLP. Our
  main observation is that, except for symbolic formula representation tasks, MLP
  generally outperforms KAN. We also conduct ablation studies on KAN and find
  that its advantage in symbolic formula representation mainly stems from its
  B-spline activation function. When B-spline is applied to MLP, performance in
  symbolic formula representation significantly improves, surpassing or matching
  that of KAN. However, in other tasks where MLP already excels over KAN,
  B-spline does not substantially enhance MLP's performance. Furthermore, we find
  that KAN's forgetting issue is more severe than that of MLP in a standard
  class-incremental continual learning setting, which differs from the findings
  reported in the KAN paper. We hope these results provide insights for future
  research on KAN and other MLP alternatives. Project link:
  https://github.com/yu-rp/KANbeFair
paper_ai_keywords: B-spline activation function class-incremental continual learning