Single-Layer Learnable Activation for Implicit Neural Representation (SL$^2$A-INR)
<|reference_start|>Single-Layer Learnable Activation for Implicit Neural Representation (SL$^2$A-INR): Implicit Neural Representation (INR), leveraging a neural network to transform coordinate input into corresponding attributes, has recently driven significant advances in several vision-related domains. However, the performance of INR is heavily influenced by the choice of the nonlinear activation function used in its multilayer perceptron (MLP) architecture. Multiple nonlinearities have been investigated; yet, current INRs face limitations in capturing high-frequency components, diverse signal types, and handling inverse problems. We have identified that these problems can be greatly alleviated by introducing a paradigm shift in INRs. We find that an architecture with learnable activations in initial layers can represent fine details in the underlying signals. Specifically, we propose SL$^{2}$A-INR, a hybrid network for INR with a single-layer learnable activation function, prompting the effectiveness of traditional ReLU-based MLPs. Our method performs superior across diverse tasks, including image representation, 3D shape reconstructions, inpainting, single image super-resolution, CT reconstruction, and novel view synthesis. Through comprehensive experiments, SL$^{2}$A-INR sets new benchmarks in accuracy, quality, and convergence rates for INR.<|reference_end|>
@article{heidari2024single-layer,
    title={Single-Layer Learnable Activation for Implicit Neural Representation
  (SL$^{2}$A-INR)},
    author={Moein Heidari, Reza Rezaeian, Reza Azad, Dorit Merhof, Hamid
  Soltanian-Zadeh, Ilker Hacihaliloglu},
    journal={arXiv preprint arXiv:2409.10836},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.10836},
    primaryClass={cs.CV}
}