In a multi-agent competitive environment, we would expect an agent who can quickly adapt to environmental changes may have a higher probability to survive and beat other agents. In this paper, to discuss whether the adaptation capability can help a learning agent to improve its competitiveness in a multi-agent environment, we construct a simplified baseball game scenario to develop and evaluate the adaptation capability of learning agents. Our baseball game scenario is modeled as a two-player zero-sum stochastic game with only the final reward. We propose a modified Deep CFR algorithm to learn the strategies of agents in a half-inning game. We also propose a strategy adaptation mechanism that continuously updates strategies based on the anticipation of the opponent's strategy in the inference time. We form several teams, with different teams adopting the same adaptation mechanism but different initial strategies, trying to analyze (1) whether an adaptation mechanism can help in increasing the winning percentage and (2) what kind of initial strategies can help a team to get a higher winning percentage. The experimental results show that the winning percentage can be increased for the team with an initial strategy learned from the modified Deep CFR algorithm. Nevertheless, those teams with deterministic initial strategies actually become less competitive.
UPDATE: Thank you for the detailed response. I think the changes you have made to the paper have improved it, but there remains significant work to complete before the paper reaches its full potential. For example, Figure 3 is a useful additional insight but it provides no quantification of the variation between repeated runs nor a comparison to a suitable baseline approach. Similarly, the additional hyperparameter details in Appendix A are helpful to enable reproducability but without explaining how these values were chosen it is impossible to assess the rigor of the empirical evaluation.
--
This paper proposes modification to Deep CFR and introduce a simplified baseball environment to evaluate the modifications. The rigour of and detail in documenting the empirical evaluation is not currently of the standard I would expect for publication at ICLR. I will detail suggestions for improvement below, but my most pressing concern for discussion in the rebuttal is with regard to the 2nd conclusion - If the team has learnt a Nash equilibrium as initial strategy, then how can performance of either team be improved by only one team further adapting their policy as shown in Table 5 lower avg2 by the difference between No Adapt (0.61) and Guest Adapt (0.51)? If I have interpreted the authors results correctly, this demonstrates that the initial policy is not a Nash equilibrium.
Suggested Improvements:
In the related work section, the narrative of the paper would be clearer if the authors introduced why the meta-learning literature is being reviewed. I would also recommend weakening the claim that "instability of the training process" is "the critical issue in MARL" to just noting it is "an issue" and request the authors justify the claim that this "is particularly severe in a competitive environment". Why is this more of an issue in fully competitive environments than it is in general sum games?
In Section 3.1 the authors claim they need to switch to TD learning from tree search due to stochastic state transitions but there are forms of search that can accommodate stochastic transitions so this claim needs to be removed. Perhaps the authors can motivate this change in another way? This section also introduces 2 further simplifications of the domain (1) learning strategies for half-innings and applying them to the whole game; (2) agents knowing their opponents true action chosen and not the noisy observation (e.g. pitchers target location instead of actual pitched location) and (3) "both agents know each other's strategies"  - these should be included in Section 2.1 when the domain is described.
Section 3.1. closes by stating "In principle, their average strategies will gradually converge to the Nash-equilibrium strategy." This is very weakly argued, to make such a claim the authors should provide evidence that their environment and modifications to Deep CFR meet the requirements of the theory where this guarantee was proven. 
Section 3.2. notes that the learning rate is "manually determined" but the precise methodology of tuning hyperparameters is not provided and no values of settings used for any hyperparameter are included in the paper. Without details of the methodolofy it is unclear if a rigorous empirical evaluation was performed and without the precise hyperparameter settings used the results are not reproducible.
Section 4.1. notes "It is very interesting to observe that the learned Nash-equilibrium strategy is actually quite similar to real-life baseball strategies." Setting aside the issue regarding whether a Nash equilibrium has been learnt, this is a subjective opinion not an rigorous empirical observation. The same applies to the comment "The simulation results are very similar to real-life baseball games" on page 9  Can you support these claims that the learnt strategy is similar to real-life by comparison to the data collected from the MLB Statcast?
In Section 4.2. there is further imprecision in the discussion of results. For example "as the strategy adaption mechanism is employed, the WP of most teams … actually decreases." As this mechanism is a core contribution of the paper, this evaluation needs to be more rigorous. Is there a statistically significant difference caused by using the proposed mechanism? 
All results presented should include quantification of variation as well as average values and the number of repeats these averages are taken from should be clearly documented. The inclusion of a limited subset of teams in empirical evaluations (e.g. Table 4 only including teams 0,1,2,3 and 12 and Table 5 only including teams 5 and 13) should be clearly justified.
All references to papers published in conference proceedings or journals should cite the published version of the paper and not the arxiv version. All references should include the full publication venue and not abbreviations (e.g. Zhang and Lesser, 2010 is currently listed as just AAAI). References to online resources should include a note of the date accessed.
Minor Comments:
1) The word "purpose" is used frequently in place of "propose" (e.g. line 7 of the abstract)
2) Page 2: "plays the-best-of-three game" -> plays the best-of-three games
3) Page 2: "that worth further study" -> that are worth further study
4) Page 3: "To simply the game" -> To simplify the game
5) Page 3: "listed in C" -> listed in Appendix C
6) Page 8 (and recurring): Initial speech marks are the wrong way around, this looks like a Latex error.
7) Page 10: "we would relief" -> we would relax
8) Page 10: "in the real life" -> in real life