hi @theudster, i believe the main reasons you don’t see cross-validation with transformers (or deep learning more generally) is because a) transfer learning is quite effective and b) most public applications involve large enough datasets where the effect of picking a bad train/test split is diminished.
having said that, nothing stops you from using cross-validation with transformers and it’s probably useful if you have a smallish number of labelled samples for fine-tuning. i’m also sure it’s heavily used in kaggle comps to get good results