In nonlinear regression, when is MLE equivalent to least squares regression? I recently received this one line question in a job interview and was a little stumped by it.

In nonlinear regression, when is Maximum Likelihood Estimation equivalent to least squares?


By definition, the least squares estimator minimises the sum of the squared distances between the actual and predicted responses.  With a set of simple steps, you can show that this estimator is equivalent to the solution of a certain maximisation problem.  If we let $f$ denote the nonlinear regression function and let $\boldsymbol{\beta}$ denote the parameter of this function (and let $\sigma>0$ be an arbitrary scaling parameter), we then have:
$$\begin{align}
\hat{\boldsymbol{\beta}}_\text{OLS}(\mathbf{y}, \mathbf{x})
&\equiv \underset{\boldsymbol{\beta}}{\text{arg min}}  \sum_{i=1}^n (y_i - f(\mathbf{x}_i, \boldsymbol{\beta}))^2 \\[6pt]
&= \underset{\boldsymbol{\beta}}{\text{arg max}} \bigg( - \sum_{i=1}^n (y_i - f(\mathbf{x}_i, \boldsymbol{\beta}))^2 \bigg) \\[6pt]
&= \underset{\boldsymbol{\beta}}{\text{arg max}} \bigg( - \frac{1}{2 \sigma^2} \sum_{i=1}^n (y_i - f(\mathbf{x}_i, \boldsymbol{\beta}))^2 \bigg) \\[6pt]
&= \underset{\boldsymbol{\beta}}{\text{arg max}} \ \exp \bigg( - \frac{1}{2 \sigma^2} \sum_{i=1}^n (y_i - f(\mathbf{x}_i, \boldsymbol{\beta}))^2 \bigg) \\[6pt]
&= \underset{\boldsymbol{\beta}}{\text{arg max}} \ \prod_{i=1}^n \exp \bigg( - \frac{1}{2 \sigma^2} (y_i - f(\mathbf{x}_i, \boldsymbol{\beta}))^2 \bigg) \\[6pt]
&= \underset{\boldsymbol{\beta}}{\text{arg max}} \ \prod_{i=1}^n \text{N} (y_i | f(\mathbf{x}_i, \boldsymbol{\beta}), \sigma^2). \\[6pt]
\end{align}$$
(These steps use the fact that the $\text{arg min}$ and $\text{arg max}$ are invariant/anti-variant to strictly monotonic transformations.  Look through the steps to ensure you understand why the minimising/maximising point is preserved under the steps.)  The latter estimator is an MLE for a certain nonlinear regression model form --- can you see what model form this is?
