Best loss function for nonlinear regression I have some nonlinear nonnormal data that I am trying to analyze. The data has been normalized to -1 to 1 and detrended with polynomials of an order of 3. I'm trying to determine if there is a special loss function for nonlinear regression type problems. I did some googling but not much showed up. Right now I'm sticking with MSE.

Other answers (like bdeonovic's and Tim's) discuss "robustness to outliers". I have to admit that while this point of view is extremely common, I do not like it very much.
I find it more helpful to think in terms of which conditional fit (or prediction) we want.

*

*Use the squared errors if you want conditional expectations as fits or predictions. ("Outliers" are then simply observations that are "far away" from the expectation, and which therefore pull the expectation towards them. If your aim is an expectation fit/prediction, then you should think long and hard about whether you want "robustness to outliers", because "outliers" are a fact of life.)

*Use the absolute errors if you want condititional medians as fits or predictions.

*Use quantile (AKA pinball) losses if you want conditional quantiles as fits or predictions.

I have written a short paper (Kolassa, 2020, IJF) on this, in the context of forecasting - but the idea holds in the precise same way for fits.
Thus, I would recommend you think about what kind of fit/prediction you want, and then tailor your loss function to this.
