Runtime and scalability of large neural networks can be significantly affected by the placement of operations in their dataflow graphs on suitable devices. With increasingly complex neural network architectures and heterogeneous device characteristics, finding a reasonable placement is extremely challenging even for domain experts. Most existing automated device placement approaches are impractical due to the significant amount of compute required and their inability to generalize to new, previously held-out graphs. To address both limitations, we propose an efficient end-to-end method based on a scalable sequential attention mechanism over a graph neural network that is transferable to new graphs. On a diverse set of representative deep learning models, including Inception-v3, AmoebaNet, Transformer-XL, and WaveNet, our method on average achieves 16% improvement over human experts and 9.2% improvement over the prior art with 15× faster convergence. To further reduce the computation cost, we pre-train the policy network on a set of dataflow graphs and use a superposition network to fine-tune it on each individual graph, achieving state-of-the-art performance on large hold-out graphs with over 50k nodes, such as an 8-layer GNMT.
GDP: Generalized Device Placement for Dataflow Graphs
This paper presents a method to assign the individual operations making up the dataflow graph of a deep neural network to a set of connected devices on which they should be executed, with the objective of maximizing runtime performance of inference. The paper’s proposed approach relies on graph neural networks to produce embeddings of the dataflow graph nodes that are claimed to be transferable to unseen graphs. These embeddings serve as input to a transformer-based sequence model that outputs the final assignment. The end-to-end model is trained with a reinforcement learning criterion minimizing the expected placement runtime. Experimental comparisons against alternative placement methods (including human expert) are presented for several large neural networks of different architectures.
Overall, the paper is well written, easy to follow, and addresses an important concern is scaling up neural networks to large model sizes. The proposed approach combining graph neural networks with transformer-based placement network appears, at first glance, novel. 
My main reservations with the paper is that it lacks many details in several key sections, preventing a full appreciation of the contributions, and making reproducibility of results impossible to contemplate. In particular:
1. How the placement network is designed and used (section 3.2) is completely lacking.
2. The details of how the training and testing datasets are obtained are also lacking. In particular, the specific model architectures for the likes of RNNLM, including hyperparameter choices, the runtime data fed to those models, etc. (This can be given in appendix). It is also not clear if the models listed in Table 1 are singular models with fixed values of hyperparameters, or if they correspond to distributions over models (e.g. with differences in hyperparameters).
3. How PPO is used is glossed over: for instance, is the proposed placement of a given model tried « live » in the inner loop of PPO to generate the reward corresponding to its runtime? What runtime data is fed to the model in order to do this? Since this would be a somewhat unusual setup, an illustration of the overall training loop would certainly solidify understanding. Moreover, a few sentences or equations explaining how PPO is used in this context would also help. Is training with PPO imperative for getting good model performance? Have other methods been tried?
In section 3.3, the parameter superposition mechanism appears to perform a kind of meta-learning. Explicit connections to such should be made, such as « Tadam: Task dependent adaptive metric for improved few-shot learning » (NeurIPS 2018).
Moreover, at the end of section 4.4, the paper claims to « report superhuman results on 8-layer GNMT ». Unless this reviewer misunderstands, the results in Table 1 (wherein GDP-one clocks in at 0.649 for 8-layer GNMT, versus 0.610 for Human Placement) would contradict this claim. This should be clarified.
Given these reservations, in spite of the potential of the proposed approach, the paper appears in its current form too immature to recommend acceptance at ICLR.