Imperfect information games are challenging benchmarks for artificial intelligent systems. To reason and plan under uncertainty is a key towards general AI. Traditionally, large amounts of simulations are used in imperfect information games, and they sometimes perform sub-optimally due to large state and action spaces. In this work, we propose a simulation reweighing mechanism using neural networks. It performs backwards verification to public previous actions and assign proper belief weights to the simulations from the information set of the current observation, using an incomplete state solver network (ISSN). We use simulation reweighing in the playing phase of the game contract bridge, and show that it outperforms previous state-of-the-art Monte Carlo simulation based methods, and achieves better play per decision.
This paper deals imperfect information games, and builds a Bayesian method to model the unknown part of the current state, making use of the past moves (which constrain the game, here Contract Bridge).
This new Bayesian method is compared to Monte Carlo style techniques, which are much more computationally expensive (they draw random samples of the unknown part of the information and then solve the perfect-information version of the game, for each simulated possible full-state).
The work also introduces a Neural Network (NN) to estimate the best moves in the perfect-information version of the game (instead of making the full tree search of the optimal moves to play).
The final model proposed (ISSN+SSN) uses a NN combined with Bayesian computation, using the NN at each time step of the past to update the belief about the current missing information.
Overall the paper is written clearly: as a non-expert in RL, I was able to follow rather easily what is done.
However, the results are not convincingly good.  Maybe it is just the interpretation/contextualization that is insufficient.  I have 3 important remarks:
1. It is stated in the abstract "that it [the new method] outperforms previous state-of-the-art Monte Carlo simulation based methods, and achieves better play per decision."  However, the costs in the 'reweighing DDS' section (table 2) are lower (better) than those of the 'reweighing SSN' section.  The results show improvement upon using some reweighing, as ISSN+sthg is better than without ISSN (although by a very small relative decrease in cost).
1.a. I understand that the NN-based methods (last three lines of table 2) are incomparably faster than the DDS-based approaches (baseline). And the total rate of tricks loss per initial state remains low (6% seems small as an absolute value).  But, they are almost double of the DDS-based loss rates !  It is not clear how this can be considered 'outperforming' or 'better play per decision'.  If you can explain in which sense those are good results for the SSN vs DDS, please do so. If they are not, please do acknowledge it, and eventually contextualize (maybe a 2-fold increase of loss rate is okay given the large speedup obtained?).
1.b. in the same way, it is not clear how ISSN outperforms its no-memory counterparts, given the loss decreases are essentially negligible. Maybe one needs to increase the value of T to make ISSN's success more obvious?
2. In addition, why not compare this Bayesian method with other Bayesian methods (quoted at the very end of section 3)?  Here the paper focuses on comparison with 'deterministic'  methods, i.e. methods which sample complete states (simulations) to then solve the complete-information version of the game (via exhaustive tree search).  Those are what I would call brute-force methods.
However, although simulations may be done (expensively) for contract bridge, in some other cases this kind of sampling may become prohibitively expensive, so that only Bayesian methods are left.  If this kind of argument is the justification for your work, please make it more explicitly.  Otherwise please correct me and explain the context (the role of Bayesian methods, what has been done and what hasn't been) more clearly.
If you are to situate the work within the Bayesian-based approaches, the question remains: is ISSN+SSN better than other Bayesian-inspired, "information-completing" methods ?  
To summarize, the paper convincingly shows that Bayesian-NN methods can compete with expensive brute force methods, but it would be very nice to see how the method introduced compares with other recent Bayesian approaches. (Or if no such comparison can be done, explain why).
3. In addition, here it seems your baseline is based on GIB, which was winning tournaments ~20 years ago: why not compare with Wbridge5 or JackBridge ?  I think you need to at least explain your choice.
Because of these weak points in terms of experimental results, I lean to reject the paper.  However, depending on the authors answers and clarifications on my 3 important remarks above, I am ready to change my rating.
Also, I have a couple of more or less minor remarks to improve the paper:
The definition of the cost is not explicitly given: "We set up the evaluation metric by tricks loss per deal. The ground truth play is DDS result for the original deal and all simulation deals. We compare all the following algorithmS with the ground truth to get costs of the policy."
You should rephrase this to make the definition of 'cost' very explicit. Is it simply the average rate of lost tricks (per given set of 13 cards in the agent hands) ?
This is quite crucial and make the reading of results a bit complicated (especially since results fo not match the conclusion announced in the abstract).
"DDS) 3 computes the maximum tricks  each side can get if all the plays are optimal"
In this place and a couple others, you should explicitly recall whether you mean 'assuming perfect information', or not.  Sometimes it can get confusing, and a bit of repetition won't hurt.  I think I understood correctly that DDS solves (perfectly) the perfect information game, but at times I thought other methods also made use of the full information (?)
Figure 4c is nicely explained and this part really illustrates well the idea of the method, I like it. Although, the notation AQ3 was not obvious for me at first, and I think it is worth improving this figure, making use of the right-hand-side space, to make it an autonomously explanatory figure.
'position': the term is not defined. I would guess it means the current state of the game (the agent's hand and the cards played in the past or during the current trick). This should be said explicitly.
Also "hands" seem to refer to the 4 hands (1 for each player) (is that correct?)
There are wrong singular/plurals ('s'/no 's') in several places. This is simple to correct and should be corrected.
in section 7, ablation studies. This is a very nice study, but you should precise how much you augment the data here (or recall by how much, if you say it earlier).
This passage is unclear and should be rephrased for clarity:
"We run DDS on all 2.4 million positions, to get the
corresponding results for each available action. This serves as ground truth of the current position.
For each position, we run 50 simulations to compute the baselines and serve as training target of the
pretrained network. Each call to DDS takes about 5 milliseconds."
This passage is unclear and should be rephrased for clarity:
"For simplicity, in this work we just use discarding information."
This passage is slightly unclear and should be rephrased for clarity:
"For each simulation, the moves with the optimal number of
tricks are marked with optimal moves. We sum up the optimal moves counter in these k simulations.
This results in a counter for each legal move and we treat this as the training target. The process is
described in Figure 1.""