chaslie:

however this causes the code to run a bit “lumpy-bumpy” & increase the GPU memoray requirements by 40% as the multi processing code is being appled to the model as well as the dataloader, is there a way which i can apply multi processing to the dataloader only?


Creating a train() method and calling it inside the if __name__ guard is the correct way.
Are you using any CUDA operations in the “global” section of the script?
If so, make sure to move them to separate functions, since only the DataLoader should use multiple workers.