On the Complexity and Approximation of Binary Evidence in Lifted Inference
Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relational models, but often resort to non-lifted inference when computing conditional probabilities. The reason is that conditioning on evidence breaks many of the model's symmetries, which can preempt standard lifting techniques. Recent theoretical results show, for example, that conditioning on evidence which corresponds to binary relations is #P-hard, suggesting that no lifting is to be expected in the worst case. In this paper, we balance this negative result by identifying the Boolean rank of the evidence as a key parameter for characterizing the complexity of conditioning in lifted inference. In particular, we show that conditioning on binary evidence with bounded Boolean rank is efficient. This opens up the possibility of approximating evidence by a low-rank Boolean matrix factorization, which we investigate both theoretically and empirically.
I should be upfront that though I am quite familiar 
      with graphical models and Markov Random Fields, I'm far from an expert on 
      Markov Logic Networks (MLNs). So I approached this paper through the lens 
      of a generic graphical models person, and inevitably I tended to interpret 
      the results in that way. This paper is interested in inference 
      (computing marginal probabilities and/or a normalization factor) in a MLN. 
      Inference is "conditional" in the sense that one will first observe some 
      "evidence", which gives a subset of allowed values to each variable or 
      pair of variables. Evidence is "unary" if it constrains single variables 
      only, and "binary" if it also constraints pairs. The main idea is the 
      following: Though inference in large MLNs is generally hard, some existing 
      "lifted" algorithms can perform inference efficiently in large models by 
      exploiting special structure in special classes of MLNs. However, 
      conditioning on binary evidence destroys this special structure. 
      This paper basically points out that, rather than conditioning on 
      binary evidence, one could augment the graph with extra variables, and 
      then condition on only on unary evidence. However, in the worst case, the 
      needed number of new variables is exponential in the number of values each 
      variable can take. However, if the binary evidence for a given pair 
      happens to be (binary) low-rank, then a much smaller number of variables 
      are needed. Thus, inference in general can be approached by (possibly 
      approximately) finding a low-rank decomposition of each binary evidence 
      term, adding extra variables, and then using the specialized unary 
      inference algorithm. The major technical result is Theorem 2 which 
      states that if you have a "lifted" inference algorithm and all pairwise 
      evidence admits a low-rank decomposition, than exact inference can be done 
      in polynomial time. This can naturally (i.e. heuristically) be done 
      approximately through approximate decomposition. A couple other 
      issues: - Is Boolean rank essential? Could one alternatively state 
      that inference is always exponential in the number of values that each 
      variable can have? (Which is weaker than Boolean rank, but simpler to 
      understand.) The experiments use a very large number of variables, which 
      motivates the decomposition, but presumably many applications have small 
      numbers, in which case decomposition is superfluous. Is this result still 
      useful? - Theorem 1 is strangely stated. I think this is previous 
      work, in which case a reference needs to be provided. I also think that 
      one should say "no polynomial time algorithm exists" rather than "takes 
      exponential time". - Finally, for what its worth, as someone quite 
      familiar with graphical models and MRFs, but not so familiar with Markov 
      logic networks, I found this paper quite challenging to understand. The 
      major issue is that very many technical terms ("evidence", "term", 
      "literal", "relation", "inference", etc.) are used undefined, and there is 
      no clear reference provided where these terms are precisely defined. After 
      reading quite a few of the background papers I think I understand these, 
      but the literature on MLNs doesn't seem totally consistent in usage, which 
      makes things a challenge. (For example, typically "evidence" is a set of 
      fixed values for a set of variables, whereas this paper uses it as a set 
      of constraints on variables or pairs of variables.) At a minimum, the 
      paper should state what is meant by inference, summarize what classes of 
      symmetries previous work has taken advantage of to make inference 
      tractable, and be precise about what evidence is and how it changes 
      inference. It could be than in an eight page paper there is no way to make 
      the current paper easily understandable by readers that aren't MLN 
      experts, but I do think more effort could be made in this direction. 
      Edit after author response: In my initial review, I 
      had a concern about how conditioning could affect the efficiency of 
      inference. After reading the other reviews, the author response, and 
      discussing with the other reviewers, I'm convinced this isn't an issue, 
      and am correspondingly raising my rating.