NEURAL GRAPH EVOLUTION: TOWARDS EFFICIENT AUTOMATIC ROBOT DESIGN
Despite the recent successes in robotic locomotion control, the design of robots, i.e., the design of their body structure, still heavily relies on human engineering. Automatic robot design has been a long studied subject, however, progress has been slow due to large combinatorial search space and the difficulty to efficiently evaluate the candidate structures. Note that one needs to both, search over many possible body structures, and choose among them based on how the robot with that structure performs in an environment. The latter means training an optimal controller given a candidate structure, which in itself is costly to obtain. In this paper, we propose Neural Graph Evolution (NGE), which performs evolutionary search in graph space, by iteratively evolving graph structures using simple mutation primitives. Key to our approach is to parameterize the control policies with graph neural networks, which allows us to transfer skills from previously evaluated designs during the graph search. This significantly reduces evaluation cost of new candidates and makes the search process orders of magnitude more efficient than that of past work. In addition, NGE applies Graph Mutation with Uncertainty (GM-UC) by incorporating model uncertainty, which reduces the search space by balancing exploration and exploitation. We show that NGE significantly outperforms previous methods in terms of convergence rate and final performance. As shown in experiments, NGE is the first algorithm that can automatically discover kinematically preferred robotic graph structures, such as a fish with two symmetric flat side-fins and a tail, or a cheetah with athletic front and back legs. NGE is extremely efficient, it finds plausible robotic structures within a day on a single 64 CPU-core Amazon EC2 machine.
This paper proposes an approach for automatic robot design based on Neural graph evolution.
The overall approach has a flavor of genetical algorithms, as it also performs evolutionary operations on the graph, but it also allows for a better mechanism for policy sharing across the different topologies, which is nice.
My main concern about the paper is that, currently, the experiments do not include any strong baseline (the ES currently is not a strong baseline, see comments below). 
The experiments currently demonstrate that optimizing both controller and hardware is better than optimizing just the controller, which is not surprising and is a phenomenon which has been previously studied in the literature.
What instead is missing is an answer to the question: Is it worth using a neural graph? what are the advantages and disadvantages compared to previous approaches?
I would like to see additional experiments to answer this questions.
In particular, I believe that any algorithms you compare against, you should optimize both G and theta, since optimizing purely the hardware is unfair.
You should use an existing ES implementation (e.g., from some well-known package) instead of a naive implementation, and as additional baseline also CMA-ES. 
If you can also compare against one or two algorithms of your choice from the recent literature it would also give more value to the comparison.
Detailed comments:
- in the abstract you say that "NGE is the first algorithm that can automatically discover complex robotic graph structures". This statement is ambiguous and potentially unsupported by evidence. how do you define complex? that can or that did discover?
- in the introduction you mention that automatic robot design had limited success. This is rather subject, and I would tend to disagree.  Moreover, the same limitations that apply to other algorithms to make them successful, in my opinion, apply to your proposed algorithm (e.g., difficulty to move from simulated to real-world).
- The digression at the bottom of the first page about neural architecture search seem out of context and interrupts the flow of the introduction. What is the point that you are trying to make? Also, note that some of the algorithms that you are citing there have indeed applied beyond architecture search, eg. Bayesian optimization is used for gait optimization in robotics, and Genetic algorithms have been used for automatic robot design.
- The stated contributions number 3 and 5 are not truly contributions. #3 is so generic that a large part of the previous literature on the topic fall under this category -- not new. #5 is weak, and tell us more about the limitations of random search and naive ES than necessarily a merit of your approach. 
- Sec 2.2: "(GNNs) are very effective" effective at what? what is the metric that you consider?
- Sec 3 "(PS), where weights are reused" can you already go into more details or refer to later sections?
- First line page 4 you mention AF, without introducing the acronym ever before.
- Sec 3.1: the statements about MB and MF algorithms are inaccurate. Model-based RL algorithms can work in real-time (e.g. http://proceedings.mlr.press/v78/drews17a/drews17a.pdf) and have been shown to have same asymptotic performance of MB controllers for simple robot control (e.g. https://arxiv.org/abs/1805.12114) 
- "to speed up and trade off between evaluating fitness and evolving new species" Unclear sentence. speed up what? why is this a trade-off?
- Sec 3.4 can you recap all the parameters after eq.11? going through Sec 3.2 and 2.2 to find them is quite annoying.
- Sec 4.1:  would argue that computational cost is rarely a concern among evolutionary algorithms. The cost of evaluating the function is typically more pressing, and as a result it is important to have algorithms that can converge within a small number of iterations/generations.
- Providing the same computational budget seem rather arbitrary at the moment, and it heavily depends from implementation. How many evaluations do you perform for each method? why not having the same budget of experiments?