This paper we present a defogger, a model that learns to predict future hidden information from partial observations. We formulate this model in the context of forward modeling and leverage spatial and sequential constraints and correlations via convolutional neural networks and long short-term memory networks, respectively. We evaluate our approach on a large dataset of human games of StarCraft: Brood War, a real-time strategy video game. Our models consistently beat strong rule-based baselines and qualitatively produce sensible future game states.
The authors introduce the task of "defogging", by which they mean attempting to infer the contents of areas in the game StarCraft hidden by "the fog of war".
The authors train a neural network to solve the defogging task, define several evaluation metrics, and argue that the neural network beats several naive baseline models. 
On the positive side, the task is a nice example of reasoning about a complex hidden state space, which is an important problem moving forwards in deep learning.
On the negative side, from what I can tell, the authors don't seem to have introduced any fundamentally new architectural choices in their neural network, so the contribution seems fairly specific to mastering StarCraft, but at the same time, the authors don't evaluate how much their defogger actually contributes to being able to win StarCraft games.  All of their evaluation is based on the accuracy of defogging. 
Granted, being able to infer hidden states is of course an important problem, but the authors appear to mainly have applied existing techniques to a benchmark that has minimal practical significance outside of being able to win StarCraft competitions, meaning that, at least as the paper is currently framed, the critical evaluation metric would be showing that a defogger helps to win games. 
Two ways I could image the contribution being improved are either highlighting and generalizing novel insights gleaned from the process of building the neural network that could help people build "defoggers" for other domains (and spelling out more explicitly what domains the authors expect their insights to generalize to), or doubling down on the StarCraft application specifically and showing that the defogger helps to win games.  A minimal version of the second modification would be having a bot that has access to a defogger play against a bot that does not have access to one.
 
All that said, as a paper on an application of deep learning, the paper appears to be solid, and if the area chairs are looking for that sort of contribution, then the work seems acceptable.
Minor points:
- Is there a benefit to having a model that jointly predicts unit presence and count, rather than having two separate models (e.g., one that feeds into the next)?  Could predicting presence or absence separately be a way to encourage sparsity, since absence of a unit is already representable as a count of zero?  The choice to have one model seems especially peculiar given the authors say they couldn't get one set of weights that works for both their classification and regression tasks
- Notation: I believe the space U is never described in the main text. What components precisely does an element of U have?
- The authors say they use gameplay from no later than 11 minutes in the game to avoid the difficulties of increasing variance. How long is a typical game?  Is this a substantial fraction of the time of the games studied?  If it is not, then perhaps the defogger would not help so much at winning.
- The F1 performance increases are somewhat small. The L1 performance gains are bigger, but the authors only compare L1 on true positives. This means they might have very bad error on false positives. (The authors state they are favoring the baseline in this comparison, but it would be nice to have those numbers.)
- I don't understand when the authors say the deep model has better memory than baselines (which includes a perfect memory baseline)