This paper addresses the problem of incremental domain adaptation (IDA) in natural language processing (NLP). We assume each domain comes one after another, and that we could only access data in the current domain. The goal of IDA is to build a unified model performing well on all the domains that we have encountered. We adopt the recurrent neural network (RNN) widely used in NLP, but augment it with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition. The memory bank provides a natural way of IDA: when adapting our model to a new domain, we progressively add new slots to the memory bank, which increases the number of parameters, and thus the model capacity. We learn the new memory slots and fine-tune existing parameters by back-propagation. Experimental results show that our approach achieves significantly better performance than fine-tuning alone. Compared with expanding hidden states, our approach is more robust for old domains, shown by both empirical and theoretical results. Our model also outperforms previous work of IDA including elastic weight consolidation and progressive neural networks in the experiments.
###Summary###
This paper introduces incremental domain adaptation for natural language processing, assuming that each domain comes one after another and only the current domain can be accessed in the application scenario.  The basic framework of this paper is based on RNN but augmented with the directly parameterized memory bank. 
The memory bank of this paper is a set of distributed, real-valued vectors capturing domain knowledge. When the model is adapted to the new domain, the model progressively increases the slots in the memory bank. 
The paper evaluates the proposed approach on an NLP classification task, i.e. multi-genre natural language inference (MultiNLI). The dataset used in this paper includes 5 genres: Slate, Fiction, Telephone, Government and Travel. 
In the experiments, the paper performs the dynamics of the progressive memory network for IDA as well as compares the proposed method with variants and previous work in the multi-domain setting. 
### Novelty ###
This paper proposes incremental domain adaptation, which is inspired by Li & Hoiem's work. The setting assumes that each domain comes one after another and only one domain can get accessed. This setting is interesting as we will encounter this setting in the real application scenarios, i.e., the domain knowledge in the real domain is unpredictable. Thus, the problem setting provides some novelty. However, I am not sure whether assuming that we can only get access to one domain is reasonable as we can always save the data for the domain we have already seen. 
From the perspective of the method, this paper incorporates the memory bank to the RNN, which is not new in the machine learning research area, but heuristic enough for the transfer learning community. 
###Clarity###
Overall, the paper is well organized and logically clear. The proposed claims are well supported by the experiments and analysis. The images are well-presented and well-explained by the captions and the text. 
###Pros###
1) The paper proposes an incremental domain adaptation scenario where one domain appears another and only the data from the current domain can be accessed, which is interesting and heuristic to the domain adaptation research community. 
2) The paper is applicable to many practical scenarios since the data from the real-world application is typically from multiple domains and the data is from one domain at a time. 
3) The paper is overall well-organized and well-written. The claims of the paper are verified by the experimental results.
###Cons###
1) The paper has a good motivation for the setting, however, one of the critical drawbacks of this paper is that the papers fail to compare with the state-of-the-art baselines. I understand that this paper has a new setting, but since the authors also compare the proposed method with the "multi-task" learning, it will be helpful to compare with state-of-the-art multi-task or multi-source baselines. 
2) The experimental results provided in this paper are weak. In Table 4, we found that sometimes, the IDA method performs worse than the multi-task baselines. 
3) The paper presents no ablation study or analysis of the experimental results. The effectiveness of the memory bank, fine-tuning/freezing learning parameters is unclear. 
It will be also interesting to see how does the proposed method performs on large-scale visual datasets.
Based on the summary, cons, and pros, the current rating I am giving now is weak reject. I would like to discuss the final rating with other reviewers, ACs. 
I am willing to improve my rating if the authors can address my following concerns. To improve the rating, the author should explain the following questions:
1). Why assuming that we can only get access to the data from one domain is a reasonable setting, since we can always save the data (or at least the statistics about the data) form the domains we have already observed.
2). Can the proposed approach generalize to visual domain adaptation, i.e. on the visual task instead of NLP task?
3). The drawbacks I mentioned in the paper Cons section.
#################### Updated Review  ###########
The authors have addressed most of my concerns I proposed in the initial review, thus I raise my score to 6 weak accept. 
1) In the response to the review, the authors claim that from the data privacy's perspective, it's reasonable to assume that we can only get access to one dataset at one time, which makes sense.
2) I would be really interested in how the similar idea works on vision data, which could be a future work for this paper.