Learning Invariant Representations to adapt deep classifiers of a source domain to a new target domain has recently attracted much attention. In this paper, we show that the search for invariance favors the compression of representations. We point out this may have a bad impact on adaptability of representations expressed as a minimal combined domain error. By considering the risk of compression, we show that weighting representations can align representation distributions without impacting their adaptability. This supports the claim that representation invariance is too strict a constraint. First, we introduce a new bound on the target risk that reveals a trade-off between compression and invariance of learned representations. More precisely, our results show that the adaptability of a representation can be better controlled when the compression risk is taken into account. In contrast, preserving adaptability may overestimate the risk of compression that makes the bound impracticable. We support these statements with a theoretical analysis illustrated on a standard domain adaptation benchmark. Second, we show that learning weighted representations plays a key role in relaxing the constraint of invariance and then preserving the risk of compression. Taking advantage of this trade-off may open up promising directions for the design of new adaptation methods.
Summary
-------
This paper presents a revisit of existing theoretical frameworks in unsupervised domain adaptation in the context of learning invariant representation. They propose a novel bound that involves trainable terms taking into account some compression information and a novel interpretation of adaptability. The authors mention also contribution showing that weighting representations can be a way to improve the analysis. 
Evaluation
-----
The ideas are novel and the result brings novel and interesting light on the difficult problem of unsupervised domain adaptation. 
However, the practical interest in terms of applicability of the proposed framework is not fully demonstrated, the properties of the proposed analysis have to be studied more in details and some parts better justified. The experimental evaluation brings some interesting behavior but is somewhat limited. The weighting aspect of the contribution is not supported by any experiment.
Other comments
------------
-I am a but puzzled by the use of the term "compression". This is maybe subjective, but in the context of learning representation, I would have interpreted it as a way to sparsify the representation, and thus compression could then be measured with respect to a given norm (L2?) or another criterion (Kolmogoroff, ...). 
In the paper, the notion of compression is related to a reduction of the hypothesis space after application of a transformation \phi, so I am wondering if using "hypothesis space reduction" would not be more appropriate.
In this case, however, there are maybe links with structural risk minimization that could be investigated here.
A side remark: there is no particular restriction on the space of transformations, we wonder if it would be useful to indicate if all the possible transformations are included as subspaces of a given latent space. Since, to be very general, one can imagine the existence of an unbounded number of transformations that correspond to an increase of the input dimension. For transformations leading to different representations of different dimensions, the way the deduced hypothesis can be compared should also be indicated (for defining properly the inclusion H(\phi_1)\subset H(\phi_2).
On the other hand, the authors seem to need the use of norms over transformations as illustrated in the definition of H_0^\eta in the experimental section. So I suggest that the analysis could be revisited by directly incorporating (representation) norms in the theoretical framework and in particular for defining more properly H_0.
-One weakness of the theoretical framework is for me the lack of definition of H_0 in Section 3. We just know that it is included between two classes of hypothesis of interest, but there is no clear characterisation of H_0 which makes the analysis fuzzy: we have a bound that involves an object without any clear definition and it is for me difficult to really interpret the bound. Trying to define H_0 with some restrictions related to the norm of the transformations, as evoked before, could be a way to address this point (and actually the way the experiments are done tend to confirm this point).
-Another weak point is the lack of qualitative analyse of the bound in Inequality 3 (the same applies for Inequality 5). I would have appreciated if the authors could provide an analysis similar to the one of (Mansour et al., COLT 2009) - it is cited in the paper - when they compared their result to the one of (Ben-David et al., 2007). For example, what happens when source is equal to the target, when is the bound significantly loose, significantly tight, different from other existing results, ...
In particular, if we compare the bound with the one of Ben-David et al. (we can also consider the one of Mansour et al.), there is two additional term, one is weighted by a factor 2, another one involved a supremum and one can think that this bound is rather loose and does not provide any insightful information and said differently it could not give a strong framework for practical considerations.
I may understand that when the bound is tight we could deduce that the compression term is low, but finding cases leading to a tight interesting bound does not seem obvious.
-The experimental evaluation presents some expected behavior in the context of the bound, but I miss a real study trying to make use of the proposed framework to do adaptation in practice with comparisons to other strategies.
Additionally, having additional studies with other models and tasks will probably reinforce the analysis.
-At the beginning of Section 3.2, the authors mention that they restrict their analysis to the square loss, however I think the analysis is true for larger class of losses with more general properties. In the experimental evaluation, the cross entropy is used, so I think that the experimental evaluation should also be consistent with the theoretical analysis by considering the square loss.
-Paragraph below Definition 5 is unclear: the notion of L2 norm has not been introduced in this context, so the message of the authors is a bit unclear.
-I do not find the notation \gamma(\phi,H) appropriate, I woud rather suggest to use \gamma(H\cdot \phi)
-The biblioggrgaphy can be improved by adding the right conferences/journals where the papers have been published in addition to the ArXiv reference.