Autoconj: Recognizing and Exploiting Conjugacy Without a Domain-Specific Language
Deriving conditional and marginal distributions using conjugacy relationships can be time consuming and error prone. In this paper, we propose a strategy for automating such derivations. Unlike previous systems which focus on relationships between pairs of random variables, our system (which we call Autoconj) operates directly on Python functions that compute log-joint distribution functions. Autoconj provides support for conjugacy-exploiting algorithms in any Pythonembedded PPL. This paves the way for accelerating development of novel inference algorithms and structure-exploiting modeling strategies.
In this paper, the authors propose a system, called Autoconj, which is able to recognize and exploit conjugacy by analyzing Python functions that compute log-joint distribution functions. Autoconj is expected to facilitate the inference of probabilistic models.   It looks like Autoconj has potential to facilitate inference of the probabilistic model, which might be very useful. The writing of the paper looks structured, but I have difficulty in understanding some details in the paper.   I do not understand the contribution of Autoconj. It looks like Autoconj takes a python function, which represents the log-joint distribution function as the input. Does this log-joint distribution function need to be properly normalized? What is the output of Autoconj? What does it mean by “recognizing and exploiting conjugacy”? Does it mean that Autoconj is able to automatically generate the marginal and conditional distribution according to the query?  If so, how are these distributions represented? Are these distributions represented in terms of a black-box pdf (i.e., given a query for a certain latent variable $z_1 = c$, Autoconj returns the approximated posterior pdf value $q(c; \eta_{z_1})$), or it returns the form and the parameters of the posterior distribution (i.e., Autoconj returns that the posterior distribution of $z_1$ is a Gaussian distribution parameterized by a specific mean and variance, for example)? In the title, it is mentioned that Autoconj is “without a Domain-Specific Language”. What does “domain-specific” mean? Why is it important to a system like Autoconj?   I also have difficulty in understanding some details about Autoconj. In Equation (3), the variable $z$ are involved. Are these variables represent the latent variables in the probabilistic model? In Equation (4) and (5), it looks like Autoconj is based on variational inference. But in line 38, it says “Autoconj is not tied to any particular approximate inference algorithms”. So is Autoconj based on variational inference approximation? Does the “canonical form” mentioned in line 130 mean Equation (3)? In Section 3.3, Autoconj analyzes the sufficient statistics for the posterior distribution. Since the joint function is not provided in a symbolic form, how does Autoconj know which statistic is relevant? How the process described in Section 3 related to the equations described in Section 2?  In addition, I typically expect that a NIPS paper involves development of new models and theorems. It looks like Autoconj is an implementation of existing models, as described in Section 2. I am also not sure whether this paper is interesting to the NIPS community.  In summary, Autoconj might be potentially useful because it facilitates inference of probabilistic models. I cannot fully understand the paper, and am not sure whether this paper would interest NIPS community. Based on the information that I can understand about this paper, I do not suggest acceptance of this paper.   I have read the author feedback. I am still not very certain how Autoconj is implemented, especially for the rewriting system. Probably more details should be included. In addition, the contribution of the paper is still not clear to me. I still do not know how this method is different from existing systems. I do not think implementing the method in python itself is a very important contribution.