Best loss function for nonlinear regression I have some nonlinear nonnormal data that I am trying to analyze. The data has been normalized to -1 to 1 and detrended with polynomials of an order of 3. I'm trying to determine if there is a special loss function for nonlinear regression type problems. I did some googling but not much showed up. Right now I'm sticking with MSE.

Most of the alternative loss functions are for making the regression more robust to outliers. I've seen all of the following in various software package implementations, but I haven't looked too hard into the literature comparing them

*

*least absolute deviation

*least median of squares

*least trimmed squares

*metric trimming

*metric winsorizing

*Huber Loss

*Tukey's biweight loss

*soft L1 loss

*Cauchy loss

*arctan loss

How are you doing the optimization? Did you code it yourself? Are you using Gaus-Newton or Gradient Descent? May want to consider Levenberg–Marquardt (interpolates between Gaus-Newton and Gradient Descent)
