A Probabilistic Framework for Multimodal Retrieval using Integrative Indian Buffet Process
We propose a multimodal retrieval procedure based on latent feature models. The procedure consists of a Bayesian nonparametric framework for learning underlying semantically meaningful abstract features in a multimodal dataset, a probabilistic retrieval model that allows cross-modal queries and an extension model for relevance feedback. Experiments on two multimodal datasets, PASCAL-Sentence and SUN-Attribute, demonstrate the effectiveness of the proposed retrieval procedure in comparison to the state-of-the-art algorithms for learning binary codes.
This paper defines a joint generative model of an image and its annotated text which is used to learn a bit vector representation for large scale image retrieval.  An Indian Buffet Process is used to learn the length of the bit vector. The method is compared favourably to several widely used techniques. Quality=======It is good to see a retrieval paper constructed around a well-defined probabilistic model.  The model is relatively simple - the assumption that X can be well approximated by ZA seems quite a strong one - it would be nice to see more discussion of the limitations imposed by this assumption. Given that Z is shared, unmodified, between the textual and visual halves of the model, it's not clear how well it would handle images with heavily occluded objects or text that doesn't mention some area of the scene. In addition, the use of a linear-Gaussian model means that the model is unimodal in visual space - which doesn't seem like a good idea. In general, there's quite a lot of discussion of the IBP which could be dropped in favour of more discussion of the model, since that is the main contribution of this paper.The experiments are thorough and show that this approach adds value over other competing approaches.  It would be nice to see more discussion about the reversal of ranking between Fig 3b and 3c.Clarity=======The paper is well written and clearly presented. It would be helpful to include a brief summary of the Euclidean neighbor ground truth labels from [6], since I found it necessary to read that paper in order to understand the differences in relative performance of the proposed method in Fig 3b and Fig 3c.Fig 2. would be made clearer by adding a plate over the images.TyposLine 038: "data-independent"->"data dependent"Line 112: "priori"->"a priori"Line 191: "As the images"->"As for the images"Line 271: "scientic"->"scientific"Originality & significance==========================The main contribution of this paper is the use of a new non-parametric Bayesian model for large scale image information retrieval that gives good performance compared to other commonly used methods.  This is a much more rigorous treatment of the problem than previous more ad-hoc, algorithmic approaches.  It also opens the doors to developing more sophisticated models in this space.  The relevance feedback model also seems like a nice contribution.