A Sampling Theory Perspective on Activations for Implicit Neural Representations
<|reference_start|>A Sampling Theory Perspective on Activations for Implicit Neural Representations: Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. While commonly using techniques like Fourier positional encodings or non-traditional activation functions (e.g., Gaussian, sinusoid, or wavelets) to capture high-frequency content, their properties lack exploration within a unified theoretical framework. Addressing this gap, we conduct a comprehensive analysis of these activations from a sampling theory perspective. Our investigation reveals that sinc activations, previously unused in conjunction with INRs, are theoretically optimal for signal encoding. Additionally, we establish a connection between dynamical systems and INRs, leveraging sampling theory to bridge these two paradigms.<|reference_end|>
@article{saratchandran2024a,
    title={A Sampling Theory Perspective on Activations for Implicit Neural
  Representations},
    author={Hemanth Saratchandran, Sameera Ramasinghe, Violetta Shevchenko,
  Alexander Long, Simon Lucey},
    journal={arXiv preprint arXiv:2402.05427},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.05427},
    primaryClass={cs.LG}
}