QoS-Nets: Adaptive Approximate Neural Network Inference
<|reference_start|>QoS-Nets: Adaptive Approximate Neural Network Inference: In order to vary the arithmetic resource consumption of neural network applications at runtime, this work proposes the flexible reuse of approximate multipliers for neural network layer computations. We introduce a search algorithm that chooses an appropriate subset of approximate multipliers of a user-defined size from a larger search space and enables retraining to maximize task performance. Unlike previous work, our approach can output more than a single, static assignment of approximate multiplier instances to layers. These different operating points allow a system to gradually adapt its Quality of Service (QoS) to changing environmental conditions by increasing or decreasing its accuracy and resource consumption. QoS-Nets achieves this by reassigning the selected approximate multiplier instances to layers at runtime. To combine multiple operating points with the use of retraining, we propose a fine-tuning scheme that shares the majority of parameters between operating points, with only a small amount of additional parameters required per operating point. In our evaluation on MobileNetV2, QoS-Nets is used to select four approximate multiplier instances for three different operating points. These operating points result in power savings for multiplications between 15.3% and 42.8% at a Top-5 accuracy loss between 0.3 and 2.33 percentage points. Through our fine-tuning scheme, all three operating points only increase the model's parameter count by only 2.75%.<|reference_end|>
@article{trommer2024qos-nets:,
    title={QoS-Nets: Adaptive Approximate Neural Network Inference},
    author={Elias Trommer, Bernd Waschneck, Akash Kumar},
    journal={arXiv preprint arXiv:2410.07762},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.07762},
    primaryClass={cs.LG}
}