Modelling and unsupervised learning of symmetric deformable object categories
We propose a new approach to model and learn, without manual supervision, the symmetries of natural objects, such as faces or flowers, given only images as input. It is well known that objects that have a symmetric structure do not usually result in symmetric images due to articulation and perspective effects. This is often tackled by seeking the intrinsic symmetries of the underlying 3D shape, which is very difficult to do when the latter cannot be recovered reliably from data. We show that, if only raw images are given, it is possible to look instead for symmetries in the space of object deformations. We can then learn symmetries from an unstructured collection of images of the object as an extension of the recently-introduced object frame representation, modified so that object symmetries reduce to the obvious symmetry groups in the normalized space. We also show that our formulation provides an explanation of the ambiguities that arise in recovering the pose of symmetric objects from their shape or images and we provide a way of discounting such ambiguities in learning.
Summary: This work propose an approach to model symmetries in deformable object categories in an unsupervised manner. This approach has been demonstrated to work for objects with bilateral symmetry (identifying symmetries in human faces using CelebA dataset, cats using cats head dataset, on cars with a synthetic car dataset), and finally for rotational symmetry on a protein structure.   Pros:  + Overview of the problem and associated challenges.  + The proposed approach seems a natural way to establish dense correspondences for non-rigid objects given two views of same object category (say example in Figure-3). In my opinion, correspondences for non-rigid/deformable objects is far more important problem than symmetry (with a potential impact on numerous problems including non rigid 3D reconstruction, wide baseline disparity estimation, human analysis etc). Reading the approach section made me actually think more in terms of correspondences than symmetry.  Cons:   - While the theory is interesting, I think the current submission is lacking on experiments. Here are my reasons for this statement:  (i)  The evaluation using CelebA face dataset cannot be termed completely "unsupervised". The face images are aligned using the 6 keypoints (left eye, right eye, nose, left side of mouth, right side of mouth, and chin) and everything is centered around nose. Most of the images have an "inherent" symmetry because of this pre-processing. Therefore, it is hard to tell the role of proposed approach in getting symmetries. I would suggest the authors to use the original CelebA dataset where faces are not aligned and show comparison.  (ii) The same argument goes for using Cats-head dataset. I would suggest to instead use Cats and Dogs dataset from Parkhi et al to demonstrate learning of symmetry. In my opinion that would be a sound approach to demonstrate the "unsupervised" part of the approach. I believe that current experiments have a hidden manual supervision and is not completely unsupervised.  (iii) This point is a suggestion to make the submission strong -- If the authors agree on (i) and (ii), than I think the most impactful experiment is demonstration of learning symmetries on ImageNet-1k dataset. Use the images from each category, and demonstrate how correspondences and symmetries naturally emerge using this approach without any manual supervision.  - An ideal comparison of this approach would be the evaluation on synthetic humans dataset (as done by Raviv et al [27]). The authors have provided a qualitative analysis in the supplementary material for synthetic humans but an extensive analysis with human study is the one that I would have loved to see. Even more interesting would have been the use of data from L. Wei et al (CVPR 2016) < http://www.hao-li.com/publications/papers/cvpr2016DHBCUCN.pdf > for demonstrating symmetries, and to verify the hypothesis of lemma-3, lemma-4, and bilateral symmetry.  Suggestion:  1. The proposed approach has a flavor of estimating symmetry or similarity via composition, so it would be good to include those work in Section-2 specifically the ones by Boiman and Irani (NIPS 2006, ICCV 2005).