@LiuzcEECS, sorry but what you say in 1. is not entirely true.  In a classifier, for each training example, each class has its own probability, but not its own loss. Considering, for instance, log-likelihood as the loss function, the loss for that example is -log§, where p denotes de the predicted probability for the true class.