josselineperdomo:

I didn’t understand what do you mean with recreate the model, could you please give me more details


I just meant to use a new instance as shown here:
model = MyModel()
# perform training on first fold

# recreate model
model = MyModel()
# perform training on second fold ...

You could of course recreate these instances in a loop, if that’s more convenient.



 josselineperdomo:

but after I tested my code I noticed I didn’t restart model’s weights when a new folds configuration starts, it is the right way to handle the model in this case?


Yes, you should reinitialize the model randomly after it was trained on a specific fold.
If you don’t want to recreate the model instance, you could call the reset_parameters() method on each submodule:
for name, module in model.named_modules():
    if hasattr(module, 'reset_parameters'):
        print('Resetting ', name)
        module.reset_parameters()

Note that I would still recommend to recreate the optimizer, as it might store running estimates (e.g. if you are using Adam).