Inference of Gradient Boosting on test istance According to my professor, Gradient Boosting can be done using the following algorithm:

Now, I do not really understand the inference part of that algorithm.
Why cannot we not simply return $F^{(K)}(x)$, which was returned in step $6$ of the algorithm? Moreover, does this method not imply that we have to keep track of each ensemble $f^{k}$ and the respective paramter $\alpha$?

*

*In gradient boosting we return the sequence of functions $F^{(k)}(x)$, $k=1, \dots, K$, $K$ being the number of trees/base learners used. $F^{(K)}(x)$ on it's own is has only learned to "correct" against the residuals done by the ensemble of the prior $K-1$ base learners. So if we returned just $F^{(K)}(x)$ only we would only do a minor addition against $F^{(0)}(x) =0$.

*In the "classic" gradient boosting paper by Friedman (2001) $\alpha$ which the  "step-length" in the the function space is indeed variable and found by line-search (the paper denotes is by $\rho$, see Eq. 8). This is indeed means that we need to keep track of it. Future implementations realised that a small but stable $\rho$ is good enough for practical purposes and making the direction of that step more relevant is how we get  a better bang of our (Stats) buck. (e.g. that's why XGBoost uses Newton's steps/second-order approximations via the Hessian instead of just the gradient.)

