Is "don't tune based on test" a small sample problem? I am trying to wrap my head around some of the principles of Machine Learning, and in particular:

Why separate test and validation sets?
The error rate estimate of the final model on validation data will be
biased (smaller than the true error rate) since the validation set is
used to select the final model After assessing the final model on the
test set, YOU MUST NOT tune the model any further! (What is the difference between test set and validation set?)

For conceptual clarity, I would like to understand whether that rule and train-validate-test tri-partition is motivated by small sample considerations, or whether there is a "more fundamental" reason to keep held-out data for unbiased measurement of model performance (part of what made me wonder is Frank Harrel's answer on the aforementioned question, which I would like to understand better, in particular in as much as it relates to small vs. large sample issues: https://stats.stackexchange.com/q/129125).
To make things more concrete, suppose we have a DGP described by $p(y,\textbf{X})$. Our job is to build a process $\pi$ which, when provided with a random sample $s_n = (y_1,\textbf{X}_1), \dots, (y_n,\textbf{X}_n)$ selects a model $\pi^{s_n}$ which associates any $\textbf{X}_i$ in the domain with a predicted value $\pi^{s_n}(\textbf{X}_i)$ for $y_i$.
Here, I am assuming that for practical reasons (think memory constraint), $n$ is fixed to some finite and "relatively low" value. So when I talk about a large sample, I am not referring to the size of $n$, but rather to the ability to generate, at least in principle, an arbitrarily large number of random samples $s^1_n, s^2_n, s^3_n, \dots$.
For concreteness, consider an imagenet-on-steroid kind of situation where my database would be made of trillions and trillions of images --- arbitrarily close to the universe of all possible human-created images. However, I can only use about 1,000,000 to decide which model to select as part of the $\pi$ procedure, again, maybe because of memory constraints.
What I care about here is the performance of the whole process. That is, on average, how well does the selected model predict new instances from the distribution according to some relevant metric. For example, I might care about $\mathbb{E}_{s^n, (\textbf{X}_i,y_i)}(|\pi^{s_n}(\textbf{X}_i) - y_i|)$.
Then, it seems the average performance over a collection of random samples $s^n$ and sets of observations $(\textbf{X}_i,y_i)$ provides an unbiased estimate of the metric I am interested in.
For example, I might compute average of my metric over 100 random sample $s^{1,000,000}_1,\dots, s^{1,000,000}_{100}$ and 100 collections of 50,000 pairs $(\textbf{X}_i,y_i)$. If I then compare two procedures $\pi_1$ and $\pi_2$ according to those averages, I am free to pick the one with the highest average metric. And my selection being based on the metric does not seem to influence the unbiasedness of the metric itself (where, again, bias is with respect to $\mathbb{E}_{s^n, (\textbf{X}_i,y_i)}(|\pi^{s_n}(\textbf{X}_i) - y_i|)$).
(In fact, variance issues aside, it seems the size of the collections of pairs $(\textbf{X}_i,y_i)$ doesn't even matter for unbiasedness?)
So here, it seems there is no need for a final held-out test-set to provide an unbiased estimate of the performance of the procedure I decided to pick? Or am I missing something?
Sorry for the long and convoluted question, and the somewhat unrealistic (?) memory constraint scenario. I hope it helps clarify the nature of my question about sample sizes, but let me know if my question remains unclear.

In low signal:noise situations as in clinical or behavioral studies, data-set splitting is risky unless you have on the order of 20,000 cases. As Harrell explains:

This is because were you to split the data again, develop a new model on the training sample, and test it on the holdout sample, the results are likely to vary significantly.

Your focus on "the whole process"

What I care about here is the performance of the whole process.

allows the following approach if you accept the bootstrap principle. You use the bootstrap to evaluate "the whole process" efficiently without completely setting aside a data subset.
First, use your process to develop the model on the entire data set.
Then repeat "the whole process" on a bootstrapped sample of the data, and see how well it works on the original full data set. Do that repeatedly starting with multiple bootstrapped samples and evaluating the model developed via "the whole process" starting from each bootstrapped sample on the full data set. Make sure that you actually repeat "the whole process" on each bootstrap sample.
Insofar as bootstrap sampling from your data set represents taking your original data set from the underlying population, you have thus evaluated how well "the whole process" might be expected to work on repeated samples from the underlying population. You can use this method to estimate the bias from the modeling process and correct the original model if desired. Perhaps you shouldn't think of this as a validation of the model per se, but it is a validation of the modeling process.
With very large data sets the split-sample approach may well be more efficient. But for those of us who work with modestly sized data sets this bootstrap validation is a more efficient use of the available data. With respect to that frequent "small sample problem," separate train/validation/test sets shouldn't be used at all.
