  Graph Hypernetworks (GHN) can predict the parameters of varying unseen CNN
architectures with surprisingly good accuracy at a fraction of the cost of
iterative optimization. Following these successes, preliminary research has
explored the use of GHNs to predict quantization-robust parameters for 8-bit
and 4-bit quantized CNNs. However, this early work leveraged full-precision
float32 training and only quantized for testing. We explore the impact of
quantization-aware training and/or other quantization-based training strategies
on quantized robustness and performance of GHN predicted parameters for
low-precision CNNs. We show that quantization-aware training can significantly
improve quantized accuracy for GHN predicted parameters of 4-bit quantized CNNs
and even lead to greater-than-random accuracy for 2-bit quantized CNNs. These
promising results open the door for future explorations such as investigating
the use of GHN predicted parameters as initialization for further quantized
training of individual CNNs, further exploration of "extreme bitwidth"
quantization, and mixed precision quantization schemes.

Machine Learning;Artificial Intelligence;Computer Vision and Pattern Recognition