As you note, this is not completely distinct. “criterion” is typically a callable (function or nn.Module instance) that computes the loss (value), “loss function” makes this explicit in the name. “loss” is - in my experience - more often used for the loss value but sometimes (for example in naming the loss Modules) it is also used for the loss function or even the class whose instance computes the loss function.
I think criterion is relatively widely understood, I have not seen ones that were not loss functions.
Loss typically is differentiable, while metrics don’t need to be. I would consider metrics to comprise both losses and non-differentiable metrics (such as accuracy). It is completely OK to use validation loss as a evaluation metric, but you cannot differentiate accuracy and so you cannot do gradient descent on it.
Best regards
Thomas