Deep Reinforcement Learning (DRL) has led to many recent breakthroughs on complex control tasks, such as defeating the best human player in the game of Go. However, decisions made by the DRL agent are not explainable, hindering its applicability in safety-critical settings. Viper, a recently proposed technique, constructs a decision tree policy by mimicking the DRL agent. Decision trees are interpretable as each action made can be traced back to the decision rule path that lead to it. However, one global decision tree approximating the DRL policy has significant limitations with respect to the geometry of decision boundaries. We propose MOËT, a more expressive, yet still interpretable model based on Mixture of Experts, consisting of a gating function that partitions the state space, and multiple decision tree experts that specialize on different partitions. We propose a training procedure to support non-differentiable decision tree experts and integrate it into imitation learning procedure of Viper. We evaluate our algorithm on four OpenAI gym environments, and show that the policy constructed in such a way is more performant and better mimics the DRL agent by lowering mispredictions and increasing the reward. We also show that MOËT policies are amenable for verification using off-the-shelf automated theorem provers such as Z3.
The paper proposes a method (MOET) to distillate a reinforcement learning policy  represented by a deep neural network into an ensemble of decision trees. The main objective of this procedure is to obtain an "interpretable" and verifiable policy while maintaining the performance of the policy. The authors build over the previously published algorithm Viper (Bastani et al, 2018), which distillates deep policies into a single decision tree using the DAGGER procedures, i.e. alternation of imitation learning of an expert policy and of additional data-sampling from the newly learned policy. In the VIPER algorithm decision trees are chosen because their structured nature allows to formally prove properties of the policy they represent when the environments dynamics are known and expressible in closed form. 
The main contribution of the paper is the adaptation of VIPER to ensembles of trees, specifically to mixtures of experts, and an adaption of the mixture of experts algorithm to non-differentiable experts. The mixture of experts is extended to decision trees with an em-like two step procedures that alternatively trains the experts and their input dependent weighting function. The main reason for extending VIPER to multiple trees is to increase the representation power of the procedure and obtain more faithful approximations of the original policy. The choice of using a input-dependent linear mixture of trees model instead of a more classical aggregation procedure like random-forests or xg-boosts is instead due to the assumed interpretability of input-dependent linear combinations.
The results in the paper seems to demonstrate comparable or improved performance of MOET over viper, but also show the systematic inability of both methods in imitating the actual neural network policy in all the environments but CARTPOLE. The authors conclude showing that the formal methods proposed to verify the policy in (Bastani et al, 2018) can be extended to MOET as they only rely on the choice of using decision trees as a target model.
On overall the paper creates a consistent narrative grounded on the notions that decision trees and linear models are interpretable, neural networks are not, individual decision trees can only create limited decision boundaries. The authors frame their technical contribution (em-like training of decision trees mixtures of experts) as necessary for creating more sophisticate decision boundaries through ensembling, while keeping the interpretability of the model (the ensembling method is linear and the components are decision trees).
While the narrative is consistent and consequent technical work seems valid the paper has some flaws that put it in my opinion slightly below the acceptance threshold:
1) The notions that linear models and decision trees have the "interpretable" propertiy is simply assumed as a known fact in the scientific field, while it is actually a largely discussed point of contention. See as an example "Lipton 2017, The Mythos of Model Interpretability". The authors should explain specifically what they mean by interpretability and how the properties of decision trees and linear mixtures of experts are consistent with their definition. 
2) On the overall paper (similarly to the narrative used for Viper) the authors speak of interpretability as if it was referred to the original deep neural network model while they only interpret the surrogate model. The lack of a formal definition of interpretable masks the problem of interpreting a model through the interpretation of a surrogate model, which would rely on some proof that links whatever is demonstrated on the surrogate to hold for the original model. 
3) The presentation of the results focus on the improvement of MOET with respect to VIPER. As a results it is difficult to compare the performance of both procedures to the original neural network model. For Pong and Acrobot the surrogate models do not achieve comparable performance, nor seems to be good imitations of the original policy, the reasons and the implication of this fact are not discussed.
4) There should be a better connection with the knowledge distillation/model compression literature and in general the important idea that the same computation can be represented by different algorithms. (Born Again Trees, Model Compression, Distilling the Knowledge in a Neural Network, Policy Distillation, Born Again Neural Networks).  There is also some related novel literature mapping neural network policies and word models to minimal hidden-markov models that could be of interest to the authors (Learning Finite State Representations of Recurrent Policy Networks, Learning Causal State Representations of Partially Observable Environments).