On the usage of many train/test split ratios Is there any benefit in evaluating model performance under different train/test ratios?
To me, this sounds kind of nonsense because of these reasons:

*

*I cannot access such a parameter when the model is in production, making it pointless to have it in a model selection process;

*How should I interpret a change from one ratio to another? I cannot say "...because the performances are stable across different ratios therefore, my pipeline is more robust.". The test sample will be of different sizes preventing any meaningful testing, especially in classification.

Of course, It may be a confirmation bias of mine. If someone has good reasons, plus some article pointing to that, I will be more than happy to read those.


any benefit in evaluating model performance under different train/test ratios?

Since different train/test ratios lead to different train set sizes, you can evaluate a somewhat special version of the model's learning curve. I.e., the performance as function of train set size.
Somewhat special here means that for small training sets the curve you get is an approximation to how your training algorithm behaves on data sets of the given size and problem - but as the split ratio approaches "all training" the curve gets closer and closer to the particular realization of one data set.
In the paper linked below you can find diagrams where we compare this with what is usually meant by learning curve (expected performance across independent data sets of given training size).
However, I've found for the type and size of data I work with that there is limited if any practically useful information to be gained from this. People try to use this technique to do sample size planning in the sense that they try to estimate what performance gain can be expected from adding more training cases. The uncertainty on this is IMHO far too large for practical purposes, though.
For details see Beleites, C. and Neugebauer, U. and Bocklitz, T. and Krafft, C. and Popp, J.: Sample size planning for classification models. Anal Chim Acta, 2013, 760, 25-33.
DOI: 10.1016/j.aca.2012.11.007
accepted manuscript on arXiv: 1211.1323

One exception may be that if you are in algorithm development rather than developing a model for production use, doing this over a number of different data sets may yield insight into how "data-hungry" the algorithm is.

You'd want to evaluate a number of splits per split ratio in any case to get an idea of how stable the behaviour for that split size is.
