What is the benefit of k-fold cross validation? I understand that using 100% of the dataset and doing k-fold cross validation instead of train_test_split would eliminate that randomness the latter method have in splitting and thus potentially avoid overfitting.
But I have seen that this is not the best practice when it comes to k-fold CV. What I have seen is that we should split the dataset (say 80% training, 20% testing), and then perform k-fold CV on the 80% training (Image below).
What I am confused about is that with this method, we are again potenially falling into a random split by using train_test_split before CV. Why is this method then considered best practice generally ? What am I missing ?


K-fold cross validation is used to determine the general fit of a model for a modelling task. It is especially useful when the amount of data is limited.
Once you determine which model is best for your problem via k-fold CV, you will train the chosen model on the entire training set, and then test the model on a dataset that was never used during the modelling process. The test set should only be used once to prevent bias. This is why you need to split the data into a training and test set before k-fold CV. Otherwise you run into the problem of having a biased model evaluation.
