Under review as a conference paper at ICLR 2018 INTERPRETABLE AND PEDAGOGICAL EXAMPLES
Teachers intentionally pick the most informative examples to show their students. However, if the teacher and student are neural networks, the examples that the teacher network learns to give, although effective at teaching the student, are typically uninterpretable. We show that training the student and teacher iteratively, rather than jointly, can produce interpretable teaching strategies. We evaluate interpretability by (1) measuring the similarity of the teacher's emergent strategies to intuitive strategies in each domain and (2) conducting human experiments to evaluate how effective the teacher's strategies are at teaching humans. We show that the teacher network learns to select or generate interpretable, pedagogical examples to teach rule-based, probabilistic, boolean, and hierarchical concepts.
The authors define a novel method for creating a pair of models, a student and a teacher model, that are co-trained in a manner such that the teacher provides useful examples to the student to communicate a concept that is interpretable to people. They do this by adapting a technique from computational cognitive science called rational pedagogy. Rather than jointly optimize the student and teacher (as done previously), they have form a coupled relation between the student and teacher where each is providing a best response to the other. The authors demonstrate that their method provides interpretable samples for teaching in commonly used psychological domains and conduct human experiments to argue it can be used to teach people in a better manner than random teaching. 
Understanding how to make complex models interpretable is an extremely important problem in ML for a number of reasons (e.g., AI ethics, explainable AI). The approach proposed by the authors is an excellent first step in this direction, and they provide a convincing argument for why a previous approach (joint optimization) did not work. It is an interesting approach that builds on computational cognitive science research and the authors provide strong evidence their method creates interpretable examples. They second part of their article, where they test the examples created by their models using behavioral experiments was less convincing. This is because they used the wrong statistical tests for analyzing the studies and it is unclear whether their results would stand with proper tests (I hope they will! – it seems clear that random samples will be harder to learn from eventually, but I also hoped there was a stronger baseline.).
For analysis, the authors use t-tests directly on KL-divergence and accuracy scores; however, this is inappropriate (see Jaeger, 2008; Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434-446.). This is especially applicable to the accuracy score results and the authors should reanalyze their data following the paper referenced above. With respect to KL-divergence, a G-test can be used (see https://en.wikipedia.org/wiki/G-test#Relation_to_Kullback.E2.80.93Leibler_divergence). I suspect the results will still be meaningful, but the appropriate analysis is essential to be able to interpret the human results.
Also, a related article: One article testing rational pedagogy in more ML contexts and using it to train ML models that is
Ho, M. K., Littman, M., MacGlashan, J., Cushman, F., & Austerweil, J. L. (NIPS 2016). Showing versus Doing. Teaching by Demonstration.
For future work, it would be nice to show that the technique works for finding interpretable examples in more complex deep learning networks, which motivated the current push for explainable AI in the first place.