Percentage change in RMSE (or MAE) over models Let's say I have two different models of an outcome Y, m1 and m2 and perform some kind of cross-validation.
I calculate the RMSE and the MAE on the test set (for the two models) and I want to say something substantively about the change in RMSE and MAE between model 1 and model 2.
Is it feasible calculate the ratio, such as RMSE_m1/RMSE_m2 or similarly (RMSE_m1-RMSE_m2)/RMSE_m1?

I find this reasonable.
Imagine a situation where some baseline model (your company’s standard) results in performance of $50$ that is considered pretty good but certainly not perfect. This performance could be RMSE, MAE, or something else.
After making some changes to the model (say you hire someone who makes more accurate measurements of important features), your new model results in performance of $45$.
I find it reasonable to describe the new model as having made a $10\%$ improvement on performance, yes.
