Why we don't use the least-square objective function for GLMs? I am familiar with OLS, in which we minimize the sum of squared residuals to obtain the estimator. Why do we not use the same approach for GLM, but instead use maximum likelihood estimation?


Why do we not use the same approach for GLM, but instead use maximum likelihood estimation?

It is because by definition GLM generalizes the OLS approach and uses more different cost functions than only the least squares method (Least squares is still included as a special case).
More precisely GLM makes two generalisations and only one of them steers away from the use of least squares.

*

*Generalisation of the functions that describe the mean. Instead of a linear function $E[Y|X] = X\beta$ we use a linear function inside another function $E[Y|X] = f(X\beta)$.
With this generalisation we can still use least squares. See for instance the fitting of an exponential function with Gaussian distributed noise (What is the objective function to optimize in glm with gaussian and poisson family?)


*Generalisation of functions that describe the conditional distribution $y|\mu_Y = f(y;\mu_Y)$ where $f$ can be a one-parameter exponential dispersion family.
It is here where the discrepancy occurs. When $f$ would be a Gaussian distribution, then the maximum likelihood estimate is equivalent to the least squares estimate.

Possibly your question is not 'why is GLM different' (it is different by definition) but instead 'why do we use it'.
The reason why we use GLM with different distributions and associated maximum likelihood estimation, is because it improve the performance of the results (the error of the estimates will be smaller).
