How to define a classification loss function for discrete ordinal values Assume multi class classification task where we have 5 labels: 1, 2, 3, 4, 5.
For simplicity, let's assume it is the rating of movies, number of stars.
I am after a loss function which is aware of the values.
Namely for for the case $y_i = 5$ if the prediction is $\hat{y}_{i} = 3$ the loss will smaller when compared to the case $\hat{y}_{i} = 1$. Namely it will punish by how far the classifier was wrong.
Is there a neural network friendly loss function for a classification like this?
Is there such loss function in the context of ensemble of trees? SVM? Maybe something in Scikit Learn?
I found something at Ordinal Categorical Classification but I was wondering if there are more options? For instance, with quadratic punishment or something else.
The formula in Keras Ordinal Categorical Crossentropy Loss Function is given by:

$$ loss(\hat{y}, y) = (1 + w) CE(\hat{y}, y) $$
Where $w = \frac{\left|class(\hat{y}) - class(y)\right|}{k - 1}$, $CE()$ is the the cross entropy, $\hat{y}$ is the probabilities per class of the classifier, $y$ the ground truth probabilities per class and $k$ is the number of classes.
The operation $class(y)$ is basically the $\arg \max$ over the vector which gets the index of the class.

The simplest approach would be to posit that your scores are interval scaled and to re-cast your problem as a numerical prediction ("regression") problem instead of as a classification one. Then you can use the Mean Squared Error between your point prediction and the outcome.
Alternatively, if you output a full probabilistic prediction with a predicted CDF of $(\hat{F}_1, \dots,  \hat{F}_5=1)$, you can use the Continuous Ranked Probability Score (CRPS, see Gneiting & Raftery, 2007), in a discrete version: for an actual outcome $k$, the CRPS is
$$ \text{CRPS}(\hat{F},k) = \sum_{j=1}^5\big(\hat{F}_j-1(j\geq k)\big)^2. $$
