@ptrblck So the model.share_memory() enables the cached model to be shared across different processes in a single GPU instead of multiple GPUs?
model1.to(device1)
model1.shared_memory()

p1 = create_process()
p2 = create_process()

p1.model_train(model1)
p2.model_train(model1)

model2= model1.clone().to(device2)
model2.shared_memory()

p3 = create_process()
p4 = create_process()

p3.model_train(model2)
p4.model_train(model2)

May I ask if the above snippet works fine?