The neural architecture search (NAS) algorithm with reinforcement learning can be a powerful and novel framework for the automatic discovering process of neural architectures. However, its application is restricted by noncontinuous and highdimensional search spaces, which result in difficulty in optimization. To resolve these problems, we proposed NAS in embedding space (NASES), which is a novel framework. Unlike other NAS with reinforcement learning approaches that search over a discrete and high-dimensional architecture space, this approach enables reinforcement learning to search in an embedding space by using architecture encoders and decoders. The current experiment demonstrated that the performance of the final architecture network using the NASES procedure is comparable with that of other popular NAS approaches for the image classification task on CIFAR-10. The beneficial-performance and effectiveness of NASES was impressive even when only the architecture-embedding searching and pre-training controller were applied without other NAS tricks such as parameter sharing. Specifically, considerable reduction in searches was achieved by reducing the average number of searching to <100 architectures to achieve a final architecture for the NASES procedure.
The paper proposes an interesting idea to perform Neural Architecture Search: first, an auto-encoder is pre-trained to encode/decode an neural architecture to/from a continuous low-dimensional embedding space; then the decoder is fixed but the encoder is copied as an agent controller for reinforcement learning. The controller is optimized by taking actions in the embedding space. The reward is also different from previous works which usually only considered validation accuracy but this work also considers the generalization gap.
The idea is interesting, but there are some problems on both the method's and the experimental sides:
1. NAO [1] also embeds neural architectures to a continuous space. Different from NAO which applies gradient descent in the embedded space, this paper uses RL. I double that RL can work better than gradient descent in a continuous space. The paper should compare with NAO. Ideally, this paper might work better than NAO if the accuracy predictor in the NAO is not accurate, while this paper uses real accuracy as a reward for search. However, this is not soundly compared.
2. It is unreasonable to discretize continuous actions to a Bernoulli distribution. Many RL methods, such as DDPG, can handle continuous actions;
3. The paper uses Eq. 1 as a reward. It's interesting, but it's unclear why the generalization error is needed. Ablation study is required.
4. As the community makes more progresses in AutoML, a better and better (smaller and smaller) search space is used. It doesn't make much sense to compare the search time under different search spaces. Comparison under the same setting (e.g. NASBench-101) is required. 
Minors:
1. missing a number in "and T0 = epochs"
2. missing "x" in "32 32 RGB in 100 classes", and "100" should be "10"
[1] Luo, Renqian, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. "Neural architecture optimization." In Advances in neural information processing systems, pp. 7816-7827. 2018.