If I intend to train multiple small models like LeNet in parallel, do I have to create n CUDA streams with each model training function in its own with block?
Can you please give pseudocode on the structure? Say I have 4 models, each with their own dataloaders and optimizers as a list.