QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted ensembles
<|reference_start|>QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted ensembles: Existing methods for uncertainty quantification incur massive memory and compute overhead, often requiring multiple models/inferences. Hence they are impractical on ultra-low-power KB-sized TinyML devices. To reduce overhead, prior works have proposed the use of early-exit networks as ensembles to quantify uncertainty in a single forward-pass. However, they still have a prohibitive cost for tinyML. To address these challenges, we propose QUTE, a novel resource-efficient early-exit-assisted ensemble architecture optimized for tinyML models. QUTE adds additional output blocks at the final exit of the base network and distills the knowledge of early-exits into these blocks to create a diverse and lightweight ensemble architecture. Our results show that QUTE outperforms popular prior works, and improves the quality of uncertainty estimates by 6% with 3.1x lower model size on average compared to the most relevant prior work. Furthermore, we demonstrate that QUTE is also effective in detecting co-variate shifted and out-of-distribution inputs, and shows competitive performance relative to G-ODIN, a state-of-the-art generalized OOD detector.<|reference_end|>
@article{ghanathe2024qute:,
    title={QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted
  ensembles},
    author={Nikhil P Ghanathe and Steve Wilton},
    journal={arXiv preprint arXiv:2404.12599},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.12599},
    primaryClass={cs.LG cs.CV}
}