title: FAN: Fourier Analysis Networks
summary: Despite the remarkable success achieved by neural networks, particularly
  those represented by MLP and Transformer, we reveal that they exhibit potential
  flaws in the modeling and reasoning of periodicity, i.e., they tend to memorize
  the periodic data rather than genuinely understanding the underlying principles
  of periodicity. However, periodicity is a crucial trait in various forms of
  reasoning and generalization, underpinning predictability across natural and
  engineered systems through recurring patterns in observations. In this paper,
  we propose FAN, a novel network architecture based on Fourier Analysis, which
  empowers the ability to efficiently model and reason about periodic phenomena.
  By introducing Fourier Series, the periodicity is naturally integrated into the
  structure and computational processes of the neural network, thus achieving a
  more accurate expression and prediction of periodic patterns. As a promising
  substitute to multi-layer perceptron (MLP), FAN can seamlessly replace MLP in
  various models with fewer parameters and FLOPs. Through extensive experiments,
  we demonstrate the effectiveness of FAN in modeling and reasoning about
  periodic functions, and the superiority and generalizability of FAN across a
  range of real-world tasks, including symbolic formula representation, time
  series forecasting, and language modeling.
paper_authors._id: 6700c883876cd40f132c439a
paper_authors.name: Yihong Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:23:39.018Z
paper_authors._id: 6468e631b2321e47d3277ec9
paper_authors.avatarUrl: /avatars/3acd3868232067289fad69db6c32cf87.svg
paper_authors.fullname: Yihong Dong
paper_authors.type: user
paper_authors.user: dongyh
paper_authors._id: 6700c883876cd40f132c439b
paper_authors.name: Ge Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:02:09.324Z
paper_authors._id: 6530b96ef68c6e013d08b413
paper_authors.avatarUrl: /avatars/ecc2874e612d0aed357d072427faf21a.svg
paper_authors.fullname: Ge Li
paper_authors.type: user
paper_authors.user: ligechina
paper_authors._id: 6700c883876cd40f132c439c
paper_authors.name: Yongding Tao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-09T14:48:39.714Z
paper_authors._id: 664b3aebfe822b08e62357f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2N3Mu7q3hV_WxELmTbrld.jpeg
paper_authors.fullname: Yongding Tao
paper_authors.type: user
paper_authors.user: YongdingTao
paper_authors._id: 6700c883876cd40f132c439d
paper_authors.name: Xue Jiang
paper_authors._id: 6700c883876cd40f132c439e
paper_authors.name: Kechi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:24:24.072Z
paper_authors._id: 62611f7c8fd541272cfa6a2d
paper_authors.avatarUrl: /avatars/c7b3cf80a9a1c78802faa97b569aba4c.svg
paper_authors.fullname: Kechi Zhang
paper_authors.type: user
paper_authors.user: zkcpku
paper_authors._id: 6700c883876cd40f132c439f
paper_authors.name: Jia Li
paper_authors._id: 6700c883876cd40f132c43a0
paper_authors.name: Jing Su
paper_authors._id: 6700c883876cd40f132c43a1
paper_authors.name: Jun Zhang
paper_authors._id: 6700c883876cd40f132c43a2
paper_authors.name: Jingjing Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:25:02.987Z
paper_authors._id: 64dc74c83ae9693d991f456c
paper_authors.avatarUrl: /avatars/395710d328a824a8a84f4b78babe5809.svg
paper_authors.fullname: Jingjing Xu
paper_authors.type: user
paper_authors.user: Jingjingxu
paper_title: FAN: Fourier Analysis Networks
paper_summary: Despite the remarkable success achieved by neural networks, particularly
  those represented by MLP and Transformer, we reveal that they exhibit potential
  flaws in the modeling and reasoning of periodicity, i.e., they tend to memorize
  the periodic data rather than genuinely understanding the underlying principles
  of periodicity. However, periodicity is a crucial trait in various forms of
  reasoning and generalization, underpinning predictability across natural and
  engineered systems through recurring patterns in observations. In this paper,
  we propose FAN, a novel network architecture based on Fourier Analysis, which
  empowers the ability to efficiently model and reason about periodic phenomena.
  By introducing Fourier Series, the periodicity is naturally integrated into the
  structure and computational processes of the neural network, thus achieving a
  more accurate expression and prediction of periodic patterns. As a promising
  substitute to multi-layer perceptron (MLP), FAN can seamlessly replace MLP in
  various models with fewer parameters and FLOPs. Through extensive experiments,
  we demonstrate the effectiveness of FAN in modeling and reasoning about
  periodic functions, and the superiority and generalizability of FAN across a
  range of real-world tasks, including symbolic formula representation, time
  series forecasting, and language modeling.
paper_ai_summary: A new Fourier-based network architecture, FAN, efficiently models periodic phenomena with fewer parameters and demonstrates superior performance across various tasks.
paper_ai_keywords: FAN Fourier Analysis Fourier Series periodic phenomena symbolic formula representation time series forecasting language modeling