Distinction between plug-in classifiers and Empirical Risk Minimization classifiers? I am currently reading the paper Fast learning rates for plug-in classifiers under the margin condition by Audibert and Tsybakov (2014), in which the authors prove that, for binary classification problems, plug-in classifiers can achieve fast convergence rates under some assumptions on the data distribution.
More precisely, for a binary classification problem with feature space $\mathcal X = \mathbb R^d$ and output space $\mathcal Y = \{0,1\}$, and the Bayes regression function defined as $\eta(x) := \mathbb P(Y=1|X=x) $, the authors define the plug-in classifiers as
$$\hat f^{PI}(X) = \mathbf 1_{\{\hat \eta_n(X)\ge\frac 1 2\}} $$
Where $\hat \eta_n$ is an estimator of the regression function $\eta$.
A lot of the discussion throughout the paper consists in comparing the convergence rate of the excess risk of these plug-in classifiers with the excess risk of Empirical Risk Minimization classifiers, which according to Wikipedia have the following expression :
$$\hat f^{ERM} = \underset{f\in\mathcal F_n}{\mathrm{argmin}} \frac 1 n \sum_{i=1}^n L(f(X_i),y_i)$$
Where $L$ is a loss function and $\mathcal F_n$ is a class of functions for the prediction rule.
I am however quite confused because it seems to me that plug-in classifiers are in fact also Empirical Risk Minimizers : Indeed, in practice, the regression function is learned by minimizing some loss depending on the datapoints, so the process of learning the regression function and thus the plug-in classifier can be thought of as a special case of Empirical Risk Minimization.
Can someone explain to me the difference between the two ?

After reading more carefully, turns out the answer was written in plain text : The estimator $\hat \eta_n$ of the regression function $\eta$ is non-parametric, it could for instance be a Kernel Density Estimator, a k-NN Estimator, a local polynomial estimator, or many other things...
Point is, $\hat \eta_n$ is not "learned" by minimizing a loss objective, and it turns out that the two frameworks (Empirical Risk Minimization vs Plug-In) are actually quite different.
