oezguensi:

How can I train two models at the same time while another process already loaded the input data in a GPU.
Do I have to specify on which device the model should run using .to_device(cuda:x) ?


Yes, I would recommend to use a single script with the DataLoader, create multiple model, push each one to the desired device via to('cuda:id') and just pass the data to each model.
Since the training is done on different devices, it should be executed in parallel.
Your approach of running multiple scripts with CUDA_VISIBLE_DEVICES would make it unnecessary complicated to share the data between these processes.