Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
The design of neural network architectures is an important component for achieving state-of-the-art performance with machine learning systems across a broad array of tasks. Much work has endeavored to design and build architectures automatically through clever construction of a search space paired with simple learning algorithms. Recent progress has demonstrated that such meta-learning methods may exceed scalable human-invented architectures on image classification tasks. An open question is the degree to which such methods may generalize to new domains. In this work we explore the construction of meta-learning techniques for dense image prediction focused on the tasks of scene parsing, person-part segmentation, and semantic image segmentation. Constructing viable search spaces in this domain is challenging because of the multi-scale representation of visual information and the necessity to operate on high resolution imagery. Based on a survey of techniques in dense image prediction, we construct a recursive search space and demonstrate that even with efficient random search, we can identify architectures that outperform human-invented architectures and achieve state-of-the-art performance on three dense prediction tasks including 82.7% on Cityscapes (street scene parsing), 71.3% on PASCAL-Person-Part (person-part segmentation), and 87.9% on PASCAL VOC 2012 (semantic image segmentation). Additionally, the resulting architecture is more computationally efficient, requiring half the parameters and half the computational cost as previous state of the art systems.
This paper introduced a method to search multi-scale architectures for dense prediction. The authors defined a search space with 81 elementary operations. As training network for dense prediction is slow, they introduced a proxy task that is fast to evaluate and predict the performance in a large-scale training setting. The new architecture is evaluated on three dense prediction tasks: scene parsing, person-part segmentation, and semantic image segmentation.  I have several concerns about this paper: -- The authors did not analyze the correlation between training architecture with 30k iterations and training architecture with 90k iterations. This result is important to confirm the approach. -- As the architecture search is based on a random search, the authors should report the standard deviation in their results. In Figure 3 and 4, the variance of the top 50 architectures seems important. I think it could be interesting to give the results every 2k architectures to show the improvement.  -- The authors did not cite Neural Fabrics [101] and GridNet [102] that learned architecture for dense prediction. -- The authors showed that it is possible to find good architectures for dense prediction but they did not analyze or give insights why the learned architecture is better than hand-crafted architecture. What is missing in the existing architectures?  What I like: + The definition of the proxy task to speed-up the architecture search + The good results on three dense prediction tasks with a more computationally efficient architecture that requires half the parameters and half the computational cost.  Other comments: - The final output of the DPC bloc is the concatenation of the of all branch outputs. Did you explore others strategies? How did you choose the number of filters? - What is the speed-up to cache the feature? - The authors should indicate the batch size.  [101] Saxena S., Verbeek J. Convolutional Neural Fabrics. In Advances in Neural Information Processing Systems, 2016. [102] Fourure D., Emonet R., Fromont E., Muselet D., Tremeau A., Wolf C. Residual Conv-Deconv Grid Network for Semantic Segmentation. In BMVC, 2017.   I have read the author response and I am globally satisfied. I think an analysis to explain what is wrong in hand crafted architectures could significantly improve the quality and the impact of this paper.