thanks so much for your fast response,
I did not try to run a full epoch with 0 workers, because it is really slow. I will try to identify exactly where this happens in the code, and I will try to run a full epoch with 0 workers.
I think the dataset is fine because this does not seem to happen when I remove my custom modules.
My best guess is that since my custom scripts involve loading a module for inference, and the module is loaded in the init method of a class, this somehow interfere with the training script. Even though the loaded modules are only used after the training is completed.
thanks again, I will come back to you!
I will also try to