Hello, I have similar question about dataloader to this question 9. But in a different manner
I’m currently writing a training script of a model consisted of 3 submodels, each trained individually. Roughly, the training iteration will be like this.
for epoch in range(n_epochs):
	# train model A
	model_a_best = model_a_step()

	# train model B
	model_b_best = model_b_step()

	# train model C
	model_c_best = model_c_step()

	# validate big model (3 submodels bundled up becoming one big model)

Unfortunately, each model has their own dataset and dataloader configurations. So I need to instantiate them both before run the training for each model.
My question is, if I were to instantiate both dataset and dataloader inside each model’s _step function, how do I destroy the dataset and dataloader once I finished training the model and goes to the next one?
My initial plan was to instantiate all the datasets and dataloaders right before this training iteration, but I’m afraid that will consume a lot of resources during the iteration