Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task. The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices. Neural Architecture Search (NAS) is a Reinforcement Learning approach that has been proposed to automate architecture design. NAS has been successfully applied to generate Neural Networks that rival the best human-designed architectures. However, NAS requires sampling, constructing, and training hundreds to thousands of models to achieve well-performing architectures. This procedure needs to be executed from scratch for each new task. The application of NAS to a wide set of tasks currently lacks a way to transfer generalizable knowledge across tasks. In this paper, we present the Multitask Neural Model Search (MNMS) controller. Our goal is to learn a generalizable framework that can condition model construction on successful model searches for previously seen tasks, thus significantly speeding up the search for new tasks. We demonstrate that MNMS can conduct an automated architecture search for multiple tasks simultaneously while still learning well-performing, specialized models for each task. We then show that pre-trained MNMS controllers can transfer learning to new tasks. By leveraging knowledge from previous searches, we find that pre-trained MNMS models start from a better location in the search space and reduce search time on unseen tasks, while still discovering models that outperform published human-designed models.
The paper proposes an extension of the Neural Architecture Search approach, in which a single RNN controller is trained with RL to select hyperparameters for child networks that must perform different tasks. The architecture includes the notion of a "task embedding", that helps the controller keeping track of similarity between tasks, to facilitate transfer across related tasks.
The paper is very well written, and based on a simple but interesting idea. It also deals with core issues in current machine learning.
On the negative side, there is just one experiment, and it is somewhat limited. In the experiment, the proposed model is trained on two very different tasks (English sentiment analysis and Spanish language detection), and then asked to generalize to another English sentiment analysis task and to a Spanish sentiment analysis task. The models converge faster to high accuracy in the proposed transfer learning setup than when trained one a single task with the same architecture search strategy. Moreover, the task embedding for the new English task is closer to that of the training English task, and the same for the training/test Spanish tasks.
My main concern with the experiment is that the approach is only tested in a setup in which there is a huge difference between two classes of tasks (English vs Spanish), so the model doesn't need to learn very sophisticated task embeddings to group the tasks correctly for transfer. It would be good to see other experiments where there is less of a trivial structure distinguishing tasks, to check if transfer helps.
Also, I find it surprising that the Corpus Cine sentiment task embedding is not correlated at all with the SST sentiment task. If the controller is really learning something interesting about the nature of the tasks, I would have expected a differential effect, such that IMDB is only correlated with SST, but Corpus Cine is correlated to both the Spanish language identification task and SST. Perhaps, this is worth some discussion.
Finally, it's not clear to me why the multitask architecture was used in the experiment even when no multi-task pre-training was conducted: shouldn't the simple neural architecture search method be used in this case?
Minor points:
"diffferentiated": different?
"outputted actions": output actions
"the aim of increase the training stability": the aim of increasing training stability
Insert references for Polyak averaging and Savitzky-Golay filtering.
Figure 3: specify that the Socher 2013 result is for SST
Figure 4: does LSS stand for SST?
I'm confused by Fig. 6: why aren't the diagonal values 100%?
MNMS is referred to as MNAS in Figure 5.
For architecture search, the neuroevolution literature should also be cited (https://www.oreilly.com/ideas/neuroevolution-a-different-kind-of-deep-learning).