Deep Hyperalignment
This paper proposes Deep Hyperalignment (DHA) as a regularized, deep extension, scalable Hyperalignment (HA) method, which is well-suited for applying functional alignment to fMRI datasets with nonlinearity, high-dimensionality (broad ROI), and a large number of subjects. Unlink previous methods, DHA is not limited by a restricted fixed kernel function. Further, it uses a parametric approach, rank-m Singular Value Decomposition (SVD), and stochastic gradient descent for optimization. Therefore, DHA has a suitable time complexity for large datasets, and DHA does not require the training data when it computes the functional alignment for a new subject. Experimental studies on multi-subject fMRI analysis confirm that the DHA method achieves superior performance to other state-of-the-art HA algorithms.
Deep Hyperalignment
[Due to time constraints, I did not read the entire paper in detail.]
The problem of functional alignment of multi-subject fMRI datasets is a fundamental and difficult problem in neuroimaging. Hyperalignment is a prominent class of anatomy-free functional alignment methods. The present manuscript presents a extension to this using a deep NN kernel. I believe this to be novel, though I don’t know the literature well enough to be confident of that. While not a surprising research direction, this is a natural and promising method to consider, and the experimental results are excellent.
The methods are explained thoroughly, with detailed proofs in supplementary material. I did however find this section too dense and without a good summary of the final design and parameter choices. A figure sketching the method would have been very helpful. Related work is surveyed well. The new method could be described as an amalgam of several existing approaches. However the various elements have been combined thoughtfully in a novel way, and carefully implemented.
The experiments are impressively thorough, involving several open datasets and several competing methods, with detailed reporting of methods, including the parameters used for each of the alignment methods. Some more detail on preprocessing would have been helpful. It seems there has been some validation of parameters on the test data (“the best performance for each dataset is reported” p6), which is not ideal, however the number of parameter values tested is small and the results are so good that I feel they would stand up to a better training/validation/test split. My main concern about this section is that all of the results reported were computed by the present authors; it would be helpful to also flag any comparable results in the literature. 
The quality of the English is mostly good but with occasional problems. E.g “time complexity fairly scales with data size” is unclear, and the sentence in which it appears is repeated three times in the manuscript (including once in the abstract). Some proof-reading is in order.