I am using the Trainer class and the hyperparameter evaluation with RayTune for fine-tunning a classification model. I already have as parameters the learning rate, the warm-up, the number of epochs, the batch size, and the weight decay. Now I want to add as hyperpameters the “model”, discerning between the cased and uncased and between large and base.
I have no problems in create different version of the models and selecting the best one in the model_init function. However, as the datasets are provided as input in the transformers.Trainer class (and they have already the texts tokenised) I am not sure what is the best approach to follow.
Any suggestion would be appreciate