Should the same cross-validation method be used across multiple models? The assignment is to write a simple ML program that trains and predicts on a dataset of our choice. I want to determine the best model for my data. The response is a class (0/1). I wrote code to try different cross-validation methods (validation set, leave-one-out, and k-fold) on multiple models (linear regression, logistic regression, k-nearest neighbors, linear discriminant analysis). Per model, I report the MSE for each cross-validation method and track the lowest one. I then pick the model with the lowest tracked MSE. This is where I think I went wrong. If I am cross-validating multiple models, should I use the same cross-validation method?

There are a few factors that make it difficult to compare results from different validation methods. Two are the size of training set, size of the test set.
In general, models perform a bit better if the training set is larger. For example, leave-one-out cross validation will have a larger dataset than using a 70/20/10 train/validation/test split. So, you would expect to have better performance from leave-one-out cross validation than from a validation set.
If you're using a larger test set, you will have more precision in your estimates of model performance (accuracy, precision, area under ROC, etc). So, if you used train/validation evaluation, you will be testing a model on a smaller group of people than you would for cross validation, so it's more likely that high performance evaluated on a train/validation split is due to chance.
So, you should use the same method on all types of models. Ideally, you would also use the same exact train/test indices across all models you train. (So, don't create new cross validation indices for each model you train and evaluate)
