We present a modular neural network architecture MAIN that learns algorithms given a set of input-output examples. MAIN consists of a neural controller that interacts with a variable-length input tape and learns to compose modules together with their corresponding argument choices. Unlike previous approaches, MAIN uses a general domain-agnostic mechanism for selection of modules and their arguments. It uses a general input tape layout together with a parallel history tape to indicate most recently used locations. Finally, it uses a memoryless controller with a length-invariant self-attention based input tape encoding to allow for random access to tape locations. The MAIN architecture is trained end-to-end using reinforcement learning from a set of input-output examples. We evaluate MAIN on five algorithmic tasks and show that it can learn policies that generalizes perfectly to inputs of much longer lengths than the ones used for training.
This paper proposes a new variation of modular neural networks. Specifically, they proposed a new architecture for the neural controller that selects which module to use given the current state and history computations, and evaluated the architectures for five symbolic tasks. I'm leaning towards rejection because:
1. The story is confusing. Instead of framing algorithm induction as a program synthesis task, the paper told the story from the modular networks's view. This is quite confusing as most of work in this line of research usually assumes the modules are neural networks and are learned, while this paper uses user provided modules. This should then be framed as doing program synthesis with pre-specified primitive functions. This work hence is quite close to NPI [1]. A RL version of NPI is also recently proposed [2].
2. Lack of novelty. Continuing from the previous point, it seems it is not easy to distinguish the main idea of this work and the work of NPI. There are two differences. There's an architectural difference about how one deals with history. NPI uses a LSTM and this work uses the last time step's computation configuration as input to a FNN. It is unclear why one is better than the other? Such modification is also lack of motivation, unexplained in the paper. Another difference is about how one designs the tasks/environments. This work uses a working tape while NPI uses programming traces.
3. Lack of baseline comparisons. In the experiment section, no previous work are evaluated and compared to the proposed method, leaving one unknown how well the proposed method compares with previously known work.
That being said, there are still some merits in the paper. The currently known work for training NPI with RL [2] does not support argument predictions, while this work does. I do not know if this is due to the interface/environment that's designed for the tasks in the paper, but I recommend the authors to rewrite the story and clarify these differences more clearly. Also how general is this environment? For harder programming tasks, can one still uses tape? How scalable is this?
Minor point
- How do you use FNN to produce all actions of the controller? Do you use a different heads for different action output? Or do you use an autoregressive architecture? Have you tried both and compared?
[1] Neural Programmer-Interpreters.
[2] Learning Compositional Neural Programs with Recursive Tree Search and Planning.