As for me: I don’t see any problem except for the simple logics of XGBoosting:
With weak learner – you’re getting 1st derivative (gradient).
Further with strong lerner – you’re getting 2nd derivative (hessian). – that crossing Zero-mark illustrates the extremum of your 1st derivative. – that is the purpose of gradient-descent (find minimum of parabolic shape).
As so as “ XG Boost splits up to the maximum depth specified and prunes the tree backward and removes splits beyond which there is an only negative loss.” – it identifies these extremums in stronger learner (that serves to make more accurate predictions than previously weak-learner did). XGBoost have no problems seeing “negative values” – it uses them for the purpose.
And XGBoost is doing it in parallel (comparing with other bosting algorithms – see link above) – why do you consider the algorithm to be the cause of “slow down” ?
The main disadvantage of XGBoost, I know, that it works poorly with highly-dimensional data. But this problem is common for most algos, therefore dimensionality-reduction in advance is worthwhile anyway.
The only problem in your logics I see in your figures (if I understood you correctly)– is taking 2nd derivative from loss function. 915
But in Algorithm’s strong-lerner the derivative is taken from losses taken from weak learner (whose aim is to classify with either losses putting further these losses to stronger-lerner) => Thus from losses themselves (residuals) you need only one derivative got in stronger-lerner, as to my opinion, as to Algorithm description at the link I've left.
Yes, and I agree with previous answer, that problem could be either in your choice of loss function(s) (can make your own loss functions) or in your data preprocessing (either transformation for data preprocessing before modelling: 1. Logarithmic, 2. Square Root, 3. Inverse (1/x) -- according data nature), also possibly you could forget to scale data beforehand.
It is also hard to understand, why do you write about “interpolate instead of fit” – as so as interpolation is used in i.e. k-NN algorithm – aiming for clustering (as unsupervised method), but XGBoost aims to classification/regresson as supervised in its nature, if you didn’t change this algorithm by your own way somehow… I hesitate to comment more till will be sure you’re using Boosting algorithm for the proper purpose & in the right way.
P.S. In my 2-core 32x PC XGBoost still could not be build in Python – therefore just in Colab can use it (to feel its speed due to parallel execution on probably multi-core Colab’s host) 139