Pure vocabulary question.
How would you define and differentiate those 3 terms : Loss, Loss Function, Criterion
Is the “Loss Function” part of the larger ensemble “Criterion”, and “Loss” the value resulting from the “Loss Function” ?
I see that documentation and tutorials often talks about loss functions, but in practice code snippet often use the term criterion instead, as the variable pointing the loss function. But this word criterion is never discussed when introducing loss functions.
Are there criterions which are not loss function ?
Is “criterion” widely understood or should I always prefer “loss function” when speaking about it ?
Side question: Metrics, Evaluation Metric.
For instance, Accuracy is one of those metrics.
Is loss part of the metrics, but not the evaluation metrics, as opposed to accuracy ?