The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth
<|reference_start|>The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth: While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which do not necessarily require a large model's complexity. Motivated by the awareness of the ever-growing AI environmental impact, we propose an efficiency strategy that leverages prior knowledge transferred by large models. Simple but effective, we propose a method relying on an Entropy-bASed Importance mEtRic (EASIER) to reduce the depth of over-parametrized deep neural networks, which alleviates their computational burden. We assess the effectiveness of our method on traditional image classification setups. Our code is available at https://github.com/VGCQ/EASIER.<|reference_end|>
@article{quétu2024the,
    title={The Simpler The Better: An Entropy-Based Importance Metric To Reduce
  Neural Networks' Depth},
    author={Victor Qu'etu, Zhu Liao, Enzo Tartaglione},
    journal={arXiv preprint arXiv:2404.18949},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.18949},
    primaryClass={cs.LG}
}