Common-sense or background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, the requisite background knowledge is indirectly acquired from static corpora. We develop a new reading architecture for the dynamic integration of explicit background knowledge in NLU models. A new task-agnostic reading module provides refined word representations to a task-specific NLU architecture by processing background knowledge in the form of free-text statements, together with the taskspecific inputs. Strong performance on the tasks of document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of our approach. Analysis shows that our models learn to exploit knowledge selectively and in a semantically appropriate way.
The main emphasis of this paper is how to add background knowledge so as to improve the performance of NLU (specifically QA and NLI) systems. They adopt the sensible perspective that background knowledge might most easily be added by providing it in text format. However, in this paper, the way it is added is simply by updating word representations based on this extra text. This seems too simple to really be the right way to add background knowledge. 
In practice, the biggest win of this paper turns out to be that you can get quite a lot of value by sharing contextualized word representations between all words with the same lemma (done by linguistic preprocessing; the paper never says exactly how, not even if you read the supplementary material). This seems a useful observation which it would be easy to apply everywhere and which shows fairly large utility from a bit of linguistically sensitive matching!  As the paper notes, this type of sharing is the main delta in this paper from simply using a standard deep LSTM (which the paper claims to not work on these data sets, though I'm not quite sure couldn't be made to work with more tuning).
pp. 6-7: The main thing of note seems to be that sharing of representations between words with the same lemma (which the tables refer to as "reading" is worth a lot (3.5-6.0%), in every case rather more than use of background knowledge (typically 0.3-1.5%). A note on the QA results: The QA results are certainly good enough to be in the range of "good systems", but none of the results really push the SOTA. The best SQuAD (devset) results are shown as several percent below the SOTA. In the table the TriviaQA results are shown as beating the SOTA, and that's fair wrt published work at the time of submission, but other submissions show that all of these results are below what you get by running the DrQA (Chen et al. 2017) system off-the-shelf on TriviaQA, so the real picture is perhaps similar to SQuAD, especially since DrQA is itself now considerably below the SOTA on SQUAD. Similar remarks perhaps apply to the NLI results.
p.7 In the additional NLI results, it is interesting and valuable to note that the lemmatization and knowledge help much more when amounts of data (and the covarying dimensionality of the word vectors) is much smaller, but the fact that the ideas of this paper have quite little (or even negative) effects when run on the full data with full word vectors on top of the ESIM model again draws into question whether enough value is being achieved from the world knowledge.
Biggest question:
 - Are word embeddings powerful enough as a form of memory to store the kind of relational facts that you are accessing as background knowledge?
Minor notes:
 - The paper was very well written/edited. The only real copyediting I noticed was in the conclusion: and be used ➔ and can be used; that rely on ➔ that relies on.
 - Should reference to (Manning et al. 1999) better be to (Manning et al. 2008) since the context here appears to be IR systems?
 - On p.3 above sec 3.1: What is u? Was that meant to be z?
 - On p.8, I'm a bit suspicious of the "Is additional knowledge used?" experiment which trains with knowledge and then tests without knowledge. It's not surprising that this mismatch might hurt performance, even if the knowledge provided no incremental value over what could be gained from standard word vectors alone.
 - In the supplementary material the paper notes that the numbers are from the best result from 3 runs. This seems to me a little less good experimental practice than reporting an average of k runs, preferably for k a bit bigger than 3.