Should one use the usual splitting (Learning/Validation/Test) when using cross-validation? Say you want to tune several parameters of your model using $N$ data. What you usually do is splitting your $N$ data into 3 sets:

*

*learning set: used to build your model;

*validation set: used to select the best parameters of your model;

*test set: used to test your model with the parameters selected on the validation set.

If you are using a CV procedure, the learning set and validation set are in fact the same and thus you only have a learning set + a test set. But my problem is that a cross-validation procedure does not avoid the appearance of overfitting. Thus, you could keep the 3 sets even if one uses CV. In that way, you use CV on the learning set to learn the parameters, you determine when to stop the learning by looking at the error on the validation set and then you test your parameters on the test set. Is that correct?

I think you are confused about what cross validation is and why you use it. Quite simply, cross validation in this context is just repeated use of the learning and validation sets you describe above. Instead of splitting your data once into learning and validation you do it multiple times and take the average of your results to decrease the variance associated with randomly splitting your data.
So cross validation isn’t any different than the process you describe above and saying “does not avoid the appearance of over fitting” is incorrect
