Recently, several studies have proven the global convergence and generalization abilities of the gradient descent method for two-layer ReLU networks. Most studies especially focused on the regression problems with the squared loss function, except for a few, and the importance of the positivity of the neural tangent kernel has been pointed out. However, the performance of gradient descent on classification problems using the logistic loss function has not been well studied, and further investigation of this problem structure is possible. In this work, we demonstrate that the separability assumption using a neural tangent model is more reasonable than the positivity condition of the neural tangent kernel and provide a refined convergence analysis of the gradient descent for two-layer networks with smooth activations. A remarkable point of our result is that our convergence and generalization bounds have much better dependence on the network width in comparison to related studies. Consequently, our theory provides a generalization guarantee for less over-parameterized two-layer networks, while most studies require much higher over-parameterization.
This paper studied the generalization performance of gradient descent for training over-parameterized two-layer neural networks on classification problems. The authors proved that under a neural tangent based separability assumption, as long as the neural network width is $\Omega(\epsilon^{-1})$, the number of training examples is $\tilde\Omega(\epsilon^{-4})$, within $O(\epsilon^{-2})$ iterations GD can achieve expected $\epsilon$-classification error.
Overall this paper is well written and easy to follow. The theoretical results on the neural network width and iteration complexity are interesting. 
My major concern is that the comparison with Allen-Zhu et al and Cao & Gu seem somewhat unfair. First, Allen-Zhu et al and Cao & Gu both studied the generalization performance of GD for training multi-layer neural networks, which is fundamentally more difficult than two-layer networks. Second, they use ReLU activation functions, which brings in the nonsmoothness along the optimization trajectory. This would also make the condition on the neural network width become worse. Therefore, when claiming the advantage of the derived guarantees, the authors should clearly clarify such differences.
Another concern is that whether the derived theoretical results can be generalized to ReLU network?
When proving the generalization result, this paper takes advantage of margin-based generalization error bound. However, the generalization results in Cao & Gu are proved via applying standard empirical Rademacher complexity based generalization error bound. I would wonder which technique can give a tighter bound?
Can you provide some examples regarding which type of data can satisfy Assumption (A.4) with constant margin $\rho$?
 The authors would like to briefly discuss another data separation assumption adopted in the following papers (although this assumption is typically made for regression problem).
[1] Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via overparameterization. arXiv preprint arXiv:1811.03962, 2018b.
[2] Allen-Zhu, Z., Li, Y. and Song, Z. (2018c). On the convergence rate of training recurrent neural networks. arXiv preprint arXiv:1810.12065 .
[3] Difan Zou, Yuan Cao, Dongruo Zhou, and Quanquan Gu. Stochastic gradient descent optimizes over-parameterized deep relu networks. arXiv preprint arXiv:1811.08888, 2018.
[4] Samet Oymak and Mahdi Soltanolkotabi. Towards moderate overparameterization: global convergence guarantees for training shallow neural networks. arXiv preprint arXiv:1902.04674, 2019.
[5] Difan Zou and Quanquan Gu. An improved analysis of training over-parameterized deep neural networks. arXiv preprint arXiv:1906.04688, 2019.