Real-world Question Answering (QA) tasks often consist of thousands of words that represent many facts and entities. Existing models based on LSTMs require a large number of parameters to support external memory and do not generalize efficiently for long sequence inputs. Memory networks address these limitations by storing information to an external memory module but must examine all inputs in the memory. Hence, for longer sequence inputs, the intermediate memory components proportionally scale in size, resulting in poor inference times. We present Adaptive Memory Networks (AMN) that process input-question pairs to dynamically construct a network architecture optimized for lower inference times. AMN creates multiple memory banks that store entities from the input story to answer the questions. The model learns to reason important entities from the input text based on the question and concentrates these entities within a single memory bank. At inference, one or few banks are used, creating a tradeoff between accuracy and performance. AMN is enabled by first, a novel bank controller that makes discrete decisions with high accuracy and second, the capabilities of a dynamic framework (such as PyTorch) that allow for dynamic network sizing and efficient variable mini-batching. In our results, we demonstrate that our model learns to construct a varying number of memory banks based on task complexity and achieves faster inference times for standard bAbI tasks, and modified bAbI tasks. We solve all bAbI tasks with an average of 48% fewer entities on tasks containing excess, unrelated information.
This paper offers a very promising approach to the processing of the type of sequences we find in dialogues, somewhat in between RNNs which have problem modeling memory, and memory networks whose explicit modeling of the memory is too rigid.
To achieve that, the starting point seems to be a strength GRU that has the ability to dynamically add memory banks to the original dialogue and question sentence representations, thanks to the use of imperative DNN programming. The use of the reparametrization trick to enable global differentiability is reminiscent of an ICLR'17 paper "Learning graphical state transitions". Compared to the latter, the current paper seems to offer a more tractable architecture and optimization problem that does not require strong supervision and should be much faster to train.
Unfortunately, this is the best understanding I got from this paper, as it seems to be in such a preliminary stage that the exact operations of the SGRU are not parsable. Maybe the authors have been taken off guard by the new review process where one can no longer improve the manuscript during this 2017 review (something that had enabled a few paper to pass the 2016 review).
After a nice introduction, everything seems to fall apart in section 4, as if the authors did not have time to finish their write-up. 
- N is both the number of sentences and number of word per sentence, which does not make sense.
- i iterates over both the sentences and the words. 
The critical SGRU algorithm is impossible to parse
- The hidden vector sigma, which is usually noted h in the GRU notation, is not even defined
- The critical reset gate operation in Eq.(6) is not even explained, and modified in a way I do not understand compared to standard GRU.
- What is t? From algorithm 1 in Appendix A, it seems to correspond to looping over both sentences and words.
- The most novel and critical operation of this SGRU, to process the entities of the memory bank, is not even explained. All we get at the end of section 4.2 is " After these steps are finished, all entities are passed through the strength modified GRU (4.1) to recompute question relevance."
The algorithm in Appendix A does  not help much.  With PyTorch being so readable, I wish some source code had been made available.
Experiments reporting also contains unacceptable omissions and errors:
- The definition of 'failed task', essential for understanding, is not stated (more than 5% error)
- Reported numbers of failed tasks are erroneous: it should be 1 for DMN+ and 3 for MemN2N.
The reviewers corrections, while significant, do not seem enough to clarify the core of the paper.
Page 3: dynanet -> dynet