Neural architecture search (NAS), or automated design of neural network models, remains a very challenging meta-learning problem. Several recent works (called "one-shot" approaches) have focused on dramatically reducing NAS running time by leveraging proxy models that still provide architectures with competitive performance. In our work, we propose a new meta-learning algorithm that we call CoNAS, or Compressive sensing-based Neural Architecture Search. Our approach merges ideas from one-shot NAS approaches with iterative techniques for learning low-degree sparse Boolean polynomial functions. We validate our approach on several standard test datasets, discover novel architectures hitherto unreported, and achieve competitive (or better) results in both performance and search time compared to existing NAS approaches. Further, we provide theoretical analysis via upper bounds on the number of validation error measurements needed to perform reliable meta-learning; to our knowledge, these analysis tools are novel to the NAS literature and may be of independent interest.
This paper proposes a new algorithm for one-shot neural architecture search (NAS) via compressive sensing. The authors propose a new search strategy, as well as a slightly different search space compared to DARTS [1], ProxylessNAS [2], etc. They use architecture samples from the one-shot model evaluated with the search parameters as a surrogate of the true objective in order to speed-up the search. Afterwards, these surrogate function evaluations are used to compute Fourier coefficients which are eventually used to optimize the vector of binary parameters encoding the architecture. 
Overall, I think the proposed algorithm is interesting and of practical usefulness. However, in terms of novelty, this work seems more to be an application of Harmonica [6] to the NAS problem (with small modifications in order to make it applicable). In page 10 you state some of the differences of your method with Harmonica. I agree that the number of function evaluations you use (coming from the one-shot model) is larger and computationally less expensive to obtain, however this does not guarantee that these are a good surrogate of the true objective that NAS aims to minimize, i.e. the validation/test accuracy of final (stand-alone) architectures . The empirical evaluations of their algorithm seem to outperform/be competitive compared to other NAS methods on all benchmarks used in the paper, however only DARTS is evaluated on their search space and the other results are taken from the corresponding papers. The paper is well-written and -structured with the caveat of being more than the recommended 8 pages.
I will adjust my score depending on the authors responses concerning the following questions/issues:
1. The correlation between the architectures evaluated using the one-shot weights and retrained from scratch, seems to be of crucial importance in your method, since you directly use the one-shot weights to collect the measurements, similarly to Random Search with weight sharing [3], ENAS [4] or Bender et al. [5]. What is the correlation of these measurements with the stand-alone architectures trained from scratch using the final evaluation settings? How did you tune the p in the Bernoulli distribution during the one-shot weight updates. According to Bender et al. [5] the ScheduledDropPath probability is an important hyperparameter affecting the aforementioned correlation.
2. What is the main motivation for using 5 operations in the operation set and not 8 as in DARTS [1] for example? Does the main contribution in the competitive results come from the different search space or the search method?
3. Is there any reference or proof for the correctness of Theorem 3.2?
4. I think there are some parts that can be moved in the Supplementary, such as the pseudocode for the proposed algorithm or Figure 3, and some other parts that can be compressed, such as the Related Work section.
	
References
[1] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  DARTS: Differentiable architecture search.  In ICLR, 2019.
[2] Han Cai, Ligeng Zhu, Song Han. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In ICLR, 2019.
[3] LIAM LI, AMEET TALWALKAR. Random Search and Reproducibility for Neural Architecture Search.
[4] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean. Efficient Neural Architecture Search via Parameter Sharing. In ICML, 2018
[5] Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, Quoc Le. Understanding and Simplifying One-Shot Architecture Search. In ICML, 2018
[6] Elad Hazan, Adam Klivans, Yang Yuan. Hyperparameter Optimization: A Spectral Approach