The Randomized Dependence Coefficient
We introduce the Randomized Dependence Coefficient (RDC), a measure of nonlinear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respect to marginal distribution transformations, has low computational cost and is easy to implement: just five lines of R code, included at the end of the paper.
The authors propose a non-linear measure of dependence 
      between two random variables. This turns out to be the canonical 
      correlation between random, nonlinear projections of the variables after a 
      copula transformation which renders the marginals of the r.vs invariant to 
      linear transformations. This work details a simple but seemingly 
      powerful procedure for quantifying non-linear dependancies between random 
      variables. Part of the key idea of the work is the use of non-linear 
      random projections via the "random fourier features" of Rahimi and Recht 
      (2007). The authors compare their new randomised dependence coefficient 
      (RDC) against a whole host of other coefficients for measuring both linear 
      and non-linear dependence. They also consider the suitability of the RDC 
      for screening for variable selection in linear prediction problems. I 
      think these are useful comparisons and experiments and they allow the 
      reader to get a decent feel for the behaviour of RDC. The overall 
      exposition of the paper is clear and each component part of the procedure 
      is clearly described. In this respect the schematic diagram in figure 1 is 
      particularly useful. Similarly, the comparison between other dependence 
      coefficients combined with the empirical results is very illuminating and 
      suggests that the performance of RDC is very promising. Although 
      the authors choose to use RFFs, recently comparisons have been made with 
      other schemes for generating random features have been made. Yang et al 
      (2012) "Nystrom Method vs Random Fourier Features: A Theoretical and 
      Empirical Comparison" flesh out some more of the theoretical properties of 
      the difference between RFF and Nystrom features. One difference in 
      particular is the difference in sampling scheme: Nystrom samples randomly 
      in such a way that it takes the probability distribution of the data into 
      account. In this sense it achieves a better estimate of the underlying 
      kernel. If this is an important property of the proposed RDC, perhaps 
      looking at Nystrom features would be interesting. It also would be 
      interesting to see the effect of different types of non-linear projections 
      where RFFs are limited to shift invariant kernels (with some other 
      extensions), Nystrom features can be computed for any type of kernel 
      function. Also, Yang et al show empirically that there is a large 
      improvement in predictive performance when more random features are used - 
      it would be interesting to see what happens when k is increased. I 
      have some concerns about the theoretical treatment presented. The 
      discussion of the bound in (10) seems to be skipping over the tradeoff 
      involving k. It seems that in order to drive the LC/sqrt(k) term to be 
      small (which could be quite slow?), ||m||_F could grow very large compared 
      with n but this is hidden by referring to the dependancy as O(1/sqrt(n)). 
      As it stands I'm not sure the analysis is sufficient to complement the 
      good empirical performance. With regards to the properties of the 
      RFF approximation for linear prediction, one possibility is that the 
      approximation acts as a regulariser. For this reason, I am not sure that 
      the type of generalisation bound used for prediction algorithms are 
      completely appropriate to quantify the performance and analyse the 
      behaviour of a dependence measure. One small point: the reference 
      to Recht is incorrectly referred to as "Brecht" in the text. 
      =============== I have read the author rebuttal and I am 
      satisfied with the response. The authors should be sure to clarify the 
      constraints in the CCA optimisation problem.