  Reducing the memory footprint of Machine Learning (ML) models, particularly
Deep Neural Networks (DNNs), is essential to enable their deployment into
resource-constrained tiny devices. However, a disadvantage of DNN models is
their vulnerability to adversarial attacks, as they can be fooled by adding
slight perturbations to the inputs. Therefore, the challenge is how to create
accurate, robust, and tiny DNN models deployable on resource-constrained
embedded devices. This paper reports the results of devising a tiny DNN model,
robust to adversarial black and white box attacks, trained with an automatic
quantizationaware training framework, i.e. QKeras, with deep quantization loss
accounted in the learning loop, thereby making the designed DNNs more accurate
for deployment on tiny devices. We investigated how QKeras and an adversarial
robustness technique, Jacobian Regularization (JR), can provide a
co-optimization strategy by exploiting the DNN topology and the per layer JR
approach to produce robust yet tiny deeply quantized DNN models. As a result, a
new DNN model implementing this cooptimization strategy was conceived,
developed and tested on three datasets containing both images and audio inputs,
as well as compared its performance with existing benchmarks against various
white-box and black-box attacks. Experimental results demonstrated that on
average our proposed DNN model resulted in 8.3% and 79.5% higher accuracy than
MLCommons/Tiny benchmarks in the presence of white-box and black-box attacks on
the CIFAR-10 image dataset and a subset of the Google Speech Commands audio
dataset respectively. It was also 6.5% more accurate for black-box attacks on
the SVHN image dataset.

Machine Learning;Cryptography and Security;Performance