  We propose three novel pruning techniques to improve the cost and results of
inference-aware Differentiable Neural Architecture Search (DNAS). First, we
introduce Prunode, a stochastic bi-path building block for DNAS, which can
search over inner hidden dimensions with O(1) memory and compute complexity.
Second, we present an algorithm for pruning blocks within a stochastic layer of
the SuperNet during the search. Third, we describe a novel technique for
pruning unnecessary stochastic layers during the search. The optimized models
resulting from the search are called PruNet and establishes a new
state-of-the-art Pareto frontier for NVIDIA V100 in terms of inference latency
for ImageNet Top-1 image classification accuracy. PruNet as a backbone also
outperforms GPUNet and EfficientNet on the COCO object detection task on
inference latency relative to mean Average Precision (mAP).

Machine Learning