Hung_Nguyen:

My question is does that mean model1 is replicated on both gpu?


Yes, DataParallel  will replicate the model, scatter inputs, and gather outputs in every iteration. So, for the above code snippet, everytime you run model1_feat = model1(input_image), model1 is replicated to all devices in the forward pass.