KAN: Kolmogorov-Arnold Networks
Inspired by the Kolmogorov-Arnold representation theorem, we propose
Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer
Perceptrons (MLPs). While MLPs have fixed activation functions on nodes
("neurons"), KANs have learnable activation functions on edges ("weights").
KANs have no linear weights at all -- every weight parameter is replaced by a
univariate function parametrized as a spline. We show that this seemingly
simple change makes KANs outperform MLPs in terms of accuracy and
interpretability. For accuracy, much smaller KANs can achieve comparable or
better accuracy than much larger MLPs in data fitting and PDE solving.
Theoretically and empirically, KANs possess faster neural scaling laws than
MLPs. For interpretability, KANs can be intuitively visualized and can easily
interact with human users. Through two examples in mathematics and physics,
KANs are shown to be useful collaborators helping scientists (re)discover
mathematical and physical laws. In summary, KANs are promising alternatives for
MLPs, opening opportunities for further improving today's deep learning models
which rely heavily on MLPs.
6631c7748d7c840b14f53b43
Ziming Liu
admin_assigned
2024-05-02T08:04:26.453Z
64d59cfa8767727dffa382de
/avatars/21611131f5f9281ff9eb954939449124.svg
Ziming Liu
user
zmliu
6631c7748d7c840b14f53b44
Yixuan Wang
6631c7748d7c840b14f53b45
Sachin Vaidya
6631c7748d7c840b14f53b46
Fabian Ruehle
admin_assigned
2024-05-02T08:04:52.586Z
648f53ddd469965a66c873e3
/avatars/2383dbd4dc806ea9f601fbb98d7fd8fc.svg
Fabian Ruehle
user
Fanianrue
6631c7748d7c840b14f53b47
James Halverson
6631c7748d7c840b14f53b48
Marin Soljačić
6631c7748d7c840b14f53b49
Thomas Y. Hou
6631c7748d7c840b14f53b4a
Max Tegmark
KAN: Kolmogorov-Arnold Networks
Inspired by the Kolmogorov-Arnold representation theorem, we propose
Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer
Perceptrons (MLPs). While MLPs have fixed activation functions on nodes
("neurons"), KANs have learnable activation functions on edges ("weights").
KANs have no linear weights at all -- every weight parameter is replaced by a
univariate function parametrized as a spline. We show that this seemingly
simple change makes KANs outperform MLPs in terms of accuracy and
interpretability. For accuracy, much smaller KANs can achieve comparable or
better accuracy than much larger MLPs in data fitting and PDE solving.
Theoretically and empirically, KANs possess faster neural scaling laws than
MLPs. For interpretability, KANs can be intuitively visualized and can easily
interact with human users. Through two examples in mathematics and physics,
KANs are shown to be useful collaborators helping scientists (re)discover
mathematical and physical laws. In summary, KANs are promising alternatives for
MLPs, opening opportunities for further improving today's deep learning models
which rely heavily on MLPs.
Kolmogorov-Arnold Networks (KANs) outperform Multi-Layer Perceptrons (MLPs) in accuracy and interpretability by using learnable activation functions and spline-based weights.
Kolmogorov-Arnold Networks KANs Multi-Layer Perceptrons MLPs learnable activation functions spline-based weights neural scaling laws interpretability data fitting PDE solving mathematical laws physical laws