Saliency, Scale and Information: Towards a Unifying Theory
In this paper we present a definition for visual saliency grounded in information theory. This proposal is shown to relate to a variety of classic research contributions in scale-space theory, interest point detection, bilateral filtering, and to existing models of visual saliency. Based on the proposed definition of visual saliency, we demonstrate results competitive with the state-of-the art for both prediction of human fixations, and segmentation of salient objects. We also characterize different properties of this model including robustness to image transformations, and extension to a wide range of other data types with 3D mesh models serving as an example. Finally, we relate this proposal more generally to the role of saliency computation in visual information processing and draw connections to putative mechanisms for saliency computation in human vision.
Summary:The paper proposes a definition of visual saliency which is based onMaxima in Information Scale-Space (MISS). Connections to scale-space theory and information theory are explained, as well as to some other works. Several experiments are reported, including on fixation and salient objects in 2D, and 3D salient features of 3D meshes.  Quality: The paper is well written, and it seems technically correct.Connections between information theory and saliency are interesting (and to some extent have been explored in the past by others, although in somewhat different formulations), and combining these with scale-space theory makes a lot of sense (and, again, to some extent, has also been explored before; e.g., see below).One problem with the quality of the paper is that the technical/mathematical contribution is fairly small, and pretty much boils down to a definition of a saliency measure. Clarity:The paper is well written.Originality:It is great the authors point out the connections to the classic works on scale-space theory, and tie together scale-space theory and information theory in the context of visual saliency. There is a paper, however, by Toews and Wells ("A mutual-information scale-space for image feature detection and feature-based classification of volumetric brain images", CVPRW, 2010) which seems to diminish *some* of the novelty of the present work. May the authors please clarify the relation to (and differences from) from Toews and Wells? Significance: This paper may have some positive impact on a subset of the NIPS community.  Remark: it is always a pleasure (for me) to see a deep net fails (Figure 5 -- where the results obtained by the proposed method are clearly better than those obtained by the deep-net approach). However, perhaps the authors should add at least one counterexample? Surely there was at least one image where the SALICON output was better than the proposed method's output...