A loss function is a guide for the model to decide its path using the optimizer. So, it will try to bring some number which must correctly reflect the gap with the actual value and also (though not limited to) -
Understand Outliers, Understand the model's purpose, Model's approach, Understand the prediction type i.e. Number, Binary label etc.
I agree that this question is too vast to answer in a short text, but still, I would try to list a summary of usage which I found most of the Authors suggesting.
This might help you to start your model but must be accompanied by individual research based on scenario and data.
It might also trigger multiple WHYs and HOWs. Ask a new question Or use the already answered questions on these(there are many)
mean_squared_error Default for regression
mean_absolute_error Regression when you have outliers
mean_squared_logarithmic_error Regression. Further scaled-down the error. Use when you expect big values in your prediction
huber_loss A mid-way of MSE and MAE. 139
This function is quadratic for small values, and linear for large values
logcosh It's again a mid way to get the benefits of both MSE and MAE log(cosh(x)) is approximately equal to (x ** 2) / 2 for small x and to abs(x) - log(2) for large x. This means that 'logcosh' works mostly like the mean squared error, but will not be so strongly affected by the occasional wildly incorrect prediction.
mean_absolute_percentage_error When we are interested in % measurement, not values. e.g. while dealing with the data of scale of a country's population, % would be more important than a big number ~10000
hinge SVM. It takes care of the margin around support vector.
categorical_crossentropy Multiclass Classification - we have one target probability per class for each instance (such as one-hot vectors, e.g. [0., 0., 0., 1., 0., 0., 0., 0., 0., 0.] to represent class 3
sparse_categorical_crossentropy Multiclass Classification - we have sparse labels (i.e., for each instance, there is just a target class index, from 0 to 9 in this case), and the classes are exclusive
binary_crossentropy Use it for simple Binary Classification
Notes <IP_ADDRESS> These are the "loss" from Keras library. The Concept would be same but other libraries may use some other text variance to name these. 506