From Activation to Initialization: Scaling Insights for Optimizing Neural Fields
<|reference_start|>From Activation to Initialization: Scaling Insights for Optimizing Neural Fields: In the realm of computer vision, Neural Fields have gained prominence as a contemporary tool harnessing neural networks for signal representation. Despite the remarkable progress in adapting these networks to solve a variety of problems, the field still lacks a comprehensive theoretical framework. This article aims to address this gap by delving into the intricate interplay between initialization and activation, providing a foundational basis for the robust optimization of Neural Fields. Our theoretical insights reveal a deep-seated connection among network initialization, architectural choices, and the optimization process, emphasizing the need for a holistic approach when designing cutting-edge Neural Fields.<|reference_end|>
@article{saratchandran2024from,
    title={From Activation to Initialization: Scaling Insights for Optimizing
  Neural Fields},
    author={Hemanth Saratchandran, Sameera Ramasinghe, Simon Lucey},
    journal={arXiv preprint arXiv:2403.19205},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.19205},
    primaryClass={cs.CV cs.LG}
}