title: Single-Layer Learnable Activation for Implicit Neural Representation
    (SL^{2}A-INR)
summary: Implicit Neural Representation (INR), leveraging a neural network to
  transform coordinate input into corresponding attributes, has recently driven
  significant advances in several vision-related domains. However, the
  performance of INR is heavily influenced by the choice of the nonlinear
  activation function used in its multilayer perceptron (MLP) architecture.
  Multiple nonlinearities have been investigated; yet, current INRs face
  limitations in capturing high-frequency components, diverse signal types, and
  handling inverse problems. We have identified that these problems can be
  greatly alleviated by introducing a paradigm shift in INRs. We find that an
  architecture with learnable activations in initial layers can represent fine
  details in the underlying signals. Specifically, we propose SL^{2}A-INR, a
  hybrid network for INR with a single-layer learnable activation function,
  prompting the effectiveness of traditional ReLU-based MLPs. Our method performs
  superior across diverse tasks, including image representation, 3D shape
  reconstructions, inpainting, single image super-resolution, CT reconstruction,
  and novel view synthesis. Through comprehensive experiments, SL^{2}A-INR sets
  new benchmarks in accuracy, quality, and convergence rates for INR.
paper_authors._id: 66ea3ec47196ffbb345c2109
paper_authors.name: Moein Heidari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-18T08:23:24.402Z
paper_authors._id: 61ba19bf6122a4fd29049371
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1639586194527-noauth.jpeg
paper_authors.fullname: Moein Heidari
paper_authors.type: user
paper_authors.user: moein99
paper_authors._id: 66ea3ec47196ffbb345c210a
paper_authors.name: Reza Rezaeian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-18T08:36:58.297Z
paper_authors._id: 6486bbd15111b6b2a88e3cce
paper_authors.avatarUrl: /avatars/96da780df80980b5380e23afa1cfd69d.svg
paper_authors.fullname: Reza Rezaeian
paper_authors.type: user
paper_authors.user: Reza-R-77
paper_authors._id: 66ea3ec47196ffbb345c210b
paper_authors.name: Reza Azad
paper_authors._id: 66ea3ec47196ffbb345c210c
paper_authors.name: Dorit Merhof
paper_authors._id: 66ea3ec47196ffbb345c210d
paper_authors.name: Hamid Soltanian-Zadeh
paper_authors._id: 66ea3ec47196ffbb345c210e
paper_authors.name: Ilker Hacihaliloglu
paper_title: Single-Layer Learnable Activation for Implicit Neural Representation
    (SL^{2}A-INR)
paper_summary: Implicit Neural Representation (INR), leveraging a neural network to
  transform coordinate input into corresponding attributes, has recently driven
  significant advances in several vision-related domains. However, the
  performance of INR is heavily influenced by the choice of the nonlinear
  activation function used in its multilayer perceptron (MLP) architecture.
  Multiple nonlinearities have been investigated; yet, current INRs face
  limitations in capturing high-frequency components, diverse signal types, and
  handling inverse problems. We have identified that these problems can be
  greatly alleviated by introducing a paradigm shift in INRs. We find that an
  architecture with learnable activations in initial layers can represent fine
  details in the underlying signals. Specifically, we propose SL^{2}A-INR, a
  hybrid network for INR with a single-layer learnable activation function,
  prompting the effectiveness of traditional ReLU-based MLPs. Our method performs
  superior across diverse tasks, including image representation, 3D shape
  reconstructions, inpainting, single image super-resolution, CT reconstruction,
  and novel view synthesis. Through comprehensive experiments, SL^{2}A-INR sets
  new benchmarks in accuracy, quality, and convergence rates for INR.
paper_ai_keywords: Implicit Neural Representation (INR) nonlinear activation function multilayer perceptron (MLP) high-frequency components diverse signal types inverse problems learnable activations SL$^{2}$A-INR hybrid network ReLU-based MLPs image representation 3D shape reconstructions inpainting single image super-resolution CT reconstruction novel view synthesis accuracy quality convergence rates