Single-Layer Learnable Activation for Implicit Neural Representation
  (SL^{2}A-INR)
Implicit Neural Representation (INR), leveraging a neural network to
transform coordinate input into corresponding attributes, has recently driven
significant advances in several vision-related domains. However, the
performance of INR is heavily influenced by the choice of the nonlinear
activation function used in its multilayer perceptron (MLP) architecture.
Multiple nonlinearities have been investigated; yet, current INRs face
limitations in capturing high-frequency components, diverse signal types, and
handling inverse problems. We have identified that these problems can be
greatly alleviated by introducing a paradigm shift in INRs. We find that an
architecture with learnable activations in initial layers can represent fine
details in the underlying signals. Specifically, we propose SL^{2}A-INR, a
hybrid network for INR with a single-layer learnable activation function,
prompting the effectiveness of traditional ReLU-based MLPs. Our method performs
superior across diverse tasks, including image representation, 3D shape
reconstructions, inpainting, single image super-resolution, CT reconstruction,
and novel view synthesis. Through comprehensive experiments, SL^{2}A-INR sets
new benchmarks in accuracy, quality, and convergence rates for INR.
66ea3ec47196ffbb345c2109
Moein Heidari
admin_assigned
2024-09-18T08:23:24.402Z
61ba19bf6122a4fd29049371
https://cdn-avatars.huggingface.co/v1/production/uploads/1639586194527-noauth.jpeg
Moein Heidari
user
moein99
66ea3ec47196ffbb345c210a
Reza Rezaeian
admin_assigned
2024-09-18T08:36:58.297Z
6486bbd15111b6b2a88e3cce
/avatars/96da780df80980b5380e23afa1cfd69d.svg
Reza Rezaeian
user
Reza-R-77
66ea3ec47196ffbb345c210b
Reza Azad
66ea3ec47196ffbb345c210c
Dorit Merhof
66ea3ec47196ffbb345c210d
Hamid Soltanian-Zadeh
66ea3ec47196ffbb345c210e
Ilker Hacihaliloglu
Single-Layer Learnable Activation for Implicit Neural Representation
  (SL^{2}A-INR)
Implicit Neural Representation (INR), leveraging a neural network to
transform coordinate input into corresponding attributes, has recently driven
significant advances in several vision-related domains. However, the
performance of INR is heavily influenced by the choice of the nonlinear
activation function used in its multilayer perceptron (MLP) architecture.
Multiple nonlinearities have been investigated; yet, current INRs face
limitations in capturing high-frequency components, diverse signal types, and
handling inverse problems. We have identified that these problems can be
greatly alleviated by introducing a paradigm shift in INRs. We find that an
architecture with learnable activations in initial layers can represent fine
details in the underlying signals. Specifically, we propose SL^{2}A-INR, a
hybrid network for INR with a single-layer learnable activation function,
prompting the effectiveness of traditional ReLU-based MLPs. Our method performs
superior across diverse tasks, including image representation, 3D shape
reconstructions, inpainting, single image super-resolution, CT reconstruction,
and novel view synthesis. Through comprehensive experiments, SL^{2}A-INR sets
new benchmarks in accuracy, quality, and convergence rates for INR.
Introduction of learnable activations in INR models enhances performance across various vision tasks by improving the capture of high-frequency components and handling inverse problems.
Implicit Neural Representation (INR) nonlinear activation function multilayer perceptron (MLP) high-frequency components diverse signal types inverse problems learnable activations SL$^{2}$A-INR single-layer learnable activation function image representation 3D shape reconstructions inpainting single image super-resolution CT reconstruction novel view synthesis accuracy quality convergence rates