Unsupervised learning of object frames by dense equivariant image labelling
One of the key challenges of visual perception is to extract abstract models of 3D objects and object categories from visual measurements, which are affected by complex nuisance factors such as viewpoint, occlusion, motion, and deformations. Starting from the recent idea of viewpoint factorization, we propose a new approach that, given a large number of images of an object and no other supervision, can extract a dense object-centric coordinate frame. This coordinate frame is invariant to deformations of the images and comes with a dense equivariant labelling neural network that can map image pixels to their corresponding object coordinates. We demonstrate the applicability of this method to simple articulated objects and deformable objects such as human faces, learning embeddings from random synthetic transformations or optical flow correspondences, all without any manual supervision.
Blue565
Unsupervised object learning from dense equivariant image labelling
An impressive paper, marred by flaws in exposition, all fixable.
The aim is to construct an object representation from multiple images, with dense labelling functions (from image to object), without supervision. Experiments seem to be very successful, though the paper would be improved by citing (somewhat) comparable numerical results on the MAFL dataset. The method is conceptually simple, which is a plus. The review of related methods seems good, though I admit to not knowing the field well enough to know what has been missed.
Unfortunately, insufficient details of the method are given. In particular, there is no discussion of the relationships between the loss functions used in practice (in (4) and (5)) and those in the theoretical discussion (in page 4, first unnumbered equation and (2)).
Also, I have some concerns about the authors’ use of the terms “equivariant” and “embedding” and their definition of “distinctive”. The standard definition of “equivariant” is: A function f is equivariant with respect to group of transformations if f(gx) = gf(x) for all x and all transformations g.
(We note that the usage of “equivariant” in [30] is correct.)
In contrast, f is invariant if f(gx) = f(x) for all x and all transformations g. 
Your definition in (1) says that \Phi is invariant with respect to image deformations. So please change the manuscript title!
Note that both equivariance and invariance are a kind of “compatibility”.
The word “embedding” usually means an embedding of an object into a larger space (with technical conditions). This doesn’t seem to be how the word is used here (though, OK, the sphere is embedded in R^3). The manuscript describes the map \Phi as an embedding. Perhaps “labelling” would be better, as in the title.
The definition of “distinctive” on p4, line 124, is equivalent to: invariance of \Phi and injectivity of \Phi(x, \cdot) for every x.
On p4, line 27:  for a constant labelling, argmax is undefined, which is worrying. 
Overall the quality of the English is good, however there are several typos that should be fixed, a few of which are noted below.
The paper is similar to [30], which is arXiv 1705.02193 (May 2017, unpublished as far as I know). They have significant overlap, however both the method and the datasets are different.
—Minor comments.
p1, line 17: typo: “form”->”from”
p2, Figure 1 caption: 
- should be “Left: Given a large number of images”
- typos: “coordiante”, “waprs”
p4, first displayed equation: \alpha undefined. should be (x,g)
p5, line 142: \Phi(x,u) has become \Phi_u x without comment.
Fig 3 caption doesn’t explain clearly what the 2nd and 3rd rows are.