The fundamental task of general density estimation has been of keen interest to machine learning. Recent advances in density estimation have either: a) proposed a flexible model to estimate the conditional factors of the chain rule, p(x i x i−1 , . . .); or b) used flexible, non-linear transformations of variables of a simple base distribution. Instead, this work jointly leverages transformations of variables and autoregressive conditional models, and proposes novel methods for both. We provide a deeper understanding of our methods, showing a considerable improvement through a comprehensive study over both real world and synthetic data. Moreover, we illustrate the use of our models in outlier detection and image modeling tasks.
This paper is well constructed and written. It consists of a number of broad ideas regarding density estimation using transformations of autoregressive networks. Specifically, the authors examine models involving linear maps from past states (LAM) and recurrence relationships (RAM). 
The critical insight is that the hidden states in the LAM are not coupled allowing considerable flexibility between consecutive conditional distributions. This is at the expense of an increased number of parameters and a lack of information sharing. In contrast, the RAM transfers information between conditional densities via the coupled hidden states allowing for more constrained smooth transitions.
The authors then explored a variety of transformations designed to increase the expressiveness of LAM and RAM. The authors importantly note that one important restriction on the class of transformations is the ability to evaluate the Jacobian of the transformation efficiently. A composite of transformations coupled with the LAM/RAM networks provides a highly expressive model for modelling arbitrary joint densities but retaining interpretable conditional structure.
There is a rich variety of synthetic and real data studies which demonstrate that LAM and RAM consistently rank amongst the top models demonstrating potential utility for this class of models.
Whilst the paper provides no definitive solutions, this is not the point of the work which seeks to provide a description of a general class of potentially useful models.