Learning Non-Gaussian Multi-Index Model via Second-Order Stein's Method
We consider estimating the parametric components of semiparametric multi-index models in high dimensions. To bypass the requirements of Gaussianity or elliptical symmetry of covariates in existing methods, we propose to leverage a second-order Stein's method with score function-based corrections. We prove that our estimator achieves a near-optimal statistical rate of convergence even when the score function or the response variable is heavy-tailed. To establish the key concentration results, we develop a data-driven truncation argument that may be of independent interest. We supplement our theoretical findings with simulations.
Summary: This article studies the estimation problem of single index and multiple index models in high dimensional setting. Using the formulation of second order Stein's lemma with sparsity assumptions, they propose an estimator formulated as a solution to a sparsity constrained semi-definite programming. The statistical rate of convergence of the estimator is derived and a numerical example is given.
Overall, I find the article interesting and useful, but their emphasis on "heavy tail" case and also partly non-Gaussianity is rather intriguing. Looking at the details, the main contribution the authors claim seems to be related to an extension of an optimality result for the estimation of sparse PCA from sub-Gaussian case to essentially sub-Exponential case through a Bernstein inequality. It would be helpful to clarify this point better.
Detailed comments are summarized below.
(a) Stein's lemma: I think one of the first references to an estimator based on Stein's lemma is H\"ardle and Stoker (1989), using the so-called "method of average derivatives", which does not require Gaussianity. 
(b) Derivative condition: Normally the condition on the second derivative would be stronger than that on the first derivative, yet, the article suggests the other way around. Is there any other place to pay the price, or does it really provide more general framework?
(c) Heavy tail case: The notation of "heavy-tail" (as a pareto-like tail to most readers, I suspect, e.g. Barthe et al., 2005) is confusing and misleading in this context. Especially, I would think the heavy tail problem in economics has more to do with extremes and I doubt that the new approach is applicable in that context. Hence, I would suggest to remove the reference to the heavy-tail part, and instead make a better link to the contributions in terms of a relaxation of the tail conditions. 
(d) Moment condition: In fact, I wonder how the moment condition on the score function could be translated in terms of the standard moment conditions (and tail conditions) on the original random variables. It would be helpful to demonstrate this point, and perhaps give a condition to make those assumptions comparable.  
(e) Relation to Sparse PCA: Given the proximity to the formulation of the estimator to sparse PCA, I was imagining that a similar technique of the proofs from sparse PCA would have been used here, (except utilizing another (Berstein-type) concentration inequality), however, the relation to sparse PCA was presented as a consequence of their results. Then, in addition to the similarity to sparse PCA, could you also demonstrate a fundamental difference in techniques/considerations, if any, (other than the moment condition, or to overcome the moment condition?) required to deal with estimation problems of single index and multiple index models?  
(f) Numerical algorithm: The algorithm to compute the estimator is mentioned only in Section 5. I would suggest to include some computational details in Section 3 as well, perhaps after (3.6).
(g) Minimum eigenvalue condition: I suppose without sparsity assumption, the minimum eigenvalue (line 174, p.5) should be zero for high dimensional case when d > k. Then, instead of "Finally" to introduce this condition, which seems to suggest that they are unrelated, would the connection need to be emphasized here as well?
Minor comments and typos:
(i) line 54, p2: (... pioneered by Ker-Chau Li, ...): From the flow of the arguments, it would be natural to add the references [20,21,22] here. 
(ii) line 75, p2: I understood "SDP" as "semi-definite programming" in the end, but it would be better to be clear from the beginning here.
(iii) equation (4.1) p.4: typo.
(iv) heck references: the style of references is not coherent.
References:
Barthe, F., Cattiaux, P. and Roberto, C. (2005) Concentration for independent random variables with heavy tails, Applied Mathematics Research Express (2). 39-60.
H\"ardle, W. and Stoker, T.M. (1989) Investigating smooth multiple regression by the method of average derivatives, JASA 84 (408), 986-995.