Efficient Online Inference for Bayesian Nonparametric Relational Models
Stochastic block models characterize observed network relationships via latent community memberships. In large social networks, we expect entities to participate in multiple communities, and the number of communities to grow with the network size. We introduce a new model for these phenomena, the hierarchical Dirichlet process relational model, which allows nodes to have mixed membership in an unbounded set of communities. To allow scalable learning, we derive an online stochastic variational inference algorithm. Focusing on assortative models of undirected networks, we also propose an efficient structured mean field variational bound, and online methods for automatically pruning unused communities. Compared to state-of-the-art online learning methods for parametric relational models, we show significantly improved perplexity and link prediction accuracy for sparse networks with tens of thousands of nodes. We also showcase an analysis of LittleSis, a large network of who-knows-who at the heights of business and government.
The paper introduces a probabilistic model for 
      networks which assigns each node in the network to multiple, overlapping 
      latent communities. Inference is done using a stochastic variational 
      method and the experimental evaluations are performed on very large 
      networks. The first thing I note is that you do not cite Morup et 
      al. (2010) "Infinite multiple membership relational modelling for complex 
      networks", which in truth was the first work to perform inference for a 
      latent feature relational model on large datasets -- in effect, rendering 
      your statement on 067-068 "... these innovations allow the first..." 
      incorrect. This is a rather serious oversight, because their paper not 
      only performs large scale inference, but their method is also an MCMC 
      method, which is well-known to usually produce more accurate results than 
      variational methods. I believe the strongest contribution from 
      this paper is the application of a stochastic variational inference method 
      to a relational data model. This aspect is likely to have the highest 
      impact. This paper provides a new choice in the toolbox of probabilistic 
      inference for relational data models, and this is always highly welcome, 
      especially since the inference is shown to be viable on massive datasets. 
      It is primarily for this reason that I will recommend the paper to be 
      accepted. I also think the experiments are very well done. It is 
      clear that great care was taken to execute the large experimental setups 
      well, and that the authors thought about the various engineering choices 
      that needed to be made to get the inference to work. I don't think these 
      engineered choices are all that elegant, but having these procedures 
      (well) outlined in a paper will certainly do service to the network 
      modelling community. Also, I don't see how this inference method 
      is "online" in any sense. The pruning moves are your solution to 
      automatically infer the number of communities/features K, which is already 
      done with the other nonparametric models for relational data. Perhaps I am 
      missing something, but I think the online aspect should be removed from 
      the paper. Indeed, I don't really even see where in the paper any aspects 
      making it particularly "online" are discussed.