The emergence of language in multi-agent settings is a promising research direction to ground natural language in simulated agents. If AI would be able to understand the meaning of language through its using it, it could also transfer it to other situations flexibly. That is seen as an important step towards achieving general AI. The scope of emergent communication is so far, however, still limited. It is necessary to enhance the learning possibilities for skills associated with communication to increase the emergable complexity. We took an example from human language acquisition and the importance of the empathic connection in this process. We propose an approach to introduce the notion of empathy to multi-agent deep reinforcement learning. We extend existing approaches on referential games with an auxiliary task for the speaker to predict the listener's mind change improving the learning time. Our experiments show the high potential of this architectural element by doubling the learning speed of the test setup.
This paper starts with a conceptual claim that incorporating a notion of “empathy” in language emergency would help agents learn faster.  The paper then proposes a learning mechanism for implementing this, and looks at its empirical effect for the case of a Speaker-Listener game.
The concept at the core of the paper is thought-provoking, somewhat grounded in human communication, and it’s interesting to see how this can be translated into a learning mechanism for the multi-agent setting.   The specific implementation proposed seems reasonable at a high level, however there are many technical details missing which really hamper the paper’s message & potential scientific impact.   The results are limited to a single game, with just a pairwise comparison (with and without “empathy”), and provide a narrow view into the effectiveness of the proposed technique.
My main problem with the paper is the clarity & organization problems.  Usually I tend to be lenient on this, thinking poor writing is much easier to fix than poor science.  But in this case the problems are large enough that the paper is just too far from the standard for ICLR publication.  It also fits in 5 pages, so the authors had lots of space to write a much better paper.   I encourage them to do this for a future submission, in addition to more extensive results, because I think the ideas are worthwhile.
Specific comments:
Design of the empathy mechanism. Can you motivate why it’s reasonable to “achieve a high relation between the hidden states of both agents”? Is this necessary / sufficient for empathy?  What are alternate framings of this?  What are properties and pros/cons of this framing?
Sec.4 needs a lot more detail!   Sections Agent setup, Learning and Empathy Extension in Sec.5 should be moved to Sec.4, since they describe the method, rather than the experiments.
Sec.5 needs better clarity.
o	What are m^l_t and M^{<l}_t in Eqn 5?  Define how h_S and h_L are parameterized, and how each is trained.
o	Fig.2 gives a high-level view of the approach, but lacks important details.  Do you apply a loss at both the Speaker’s Decoder output, and the Listener’s Decoder output?  Or just the latter?  What is the loss specifically? I assume combination of Eqn 5 and Eqn 6, but not sure.
o	If you train just on the loss of the Listener’s Decoder, does this mean this is backpropagated all the way to train the Speaker?  How would this be done in a real system?  It’s a very strong assumption to say that the Listener will share gradients with the Speaker.  It seems more realistic to assume they will each observe a loss and train independently.
Results are very brief.
o	How robust are the results to the specification of the \alpha (the loss weight from Eqn 6)?  How much data goes to finding a good \alpha?
o	What is the difference between the left and the right plot?  Is one for the Speaker and the other for the Listener?
o	How do you measure “learning speed”, which is the main metric discussed in the text of Sec.6?
o	How do the results change by number of concepts in the game?
o	Do you do any pre-training of the encoder/decoder networks?
o	Can you show confidence intervals on each curve?
o	Can you show test performance?
o	Are there other related games to consider?
Many references missing throughout to support statements, e.g.
o	“Natural language is not as rule-based as…” 
o	“These considerations led to the research field of emergent communication…”
o	Sec.2:  Earlier refs to RL in general (e.g. work Sutton in the 1980’s). Earlier refs to RL with neural networks (e.g. work of G. Tesauro; work of M. Riedmiller).
o	Referential game in Fig.1 caption.
Some minor language issues, e.g.
o	“The field was then alleviated” -> Do you mean elevated?