Negative predicted probabilities with XGBRegressor I'm testing using XGBRegression instead of XGBClassifier so I tried using all three objective functions it provides: reg:logistic, reg:squarederror and reg:squaredlogerror
When I try reg:logistic the algorithm seems to work okay, the output is the probability of class 1 (since this is a binary case).
However when I try the other two objective functions the result is negative probabilities in some cases.
Why would that happen?

The function reg:logistic enforces that the predictions are probabilities. On the other hand, the functions reg:squarederror and reg:squaredlogerror don't know that you're trying to model a probability, so they don't enforce non-negativity constraints.
This is essentially the same problem that can happen with OLS regression on a binary outcome: you have a linear predictor on the probability scale, so you can predict values less than 0 or greater than 1. These predicted values can't be interpreted as "probabilities," because they do not conform to the requirements of probabilities. This phenomenon is why using OLS is discouraged when you're attempting to estimate the probability of a categorical outcome; instead, GLMs are used. Likewise, your XGBoost model should use a method which enforces that the predictions are probabilities.
