In this case I made some modifacations to the parameters of a DataParallel wrapped model.
Such as
model = resnet50()
model = nn.DataParallel(model).cuda()
# several parameter reinitialization tricks here
model.module.conv1.weight.data = xxx

In my understanding there is a copy of the original model in each GPU device.
My question is: ‘will the aforementioned modifications to the nn.DataParallel wrapped model be broadcasted to all GPUs’?
And how about if I change the architecture of the model (such as replacing a conv3x3 in the original
model with dilated convolution)?

If it won’t be broadcasted to all other GPUs, how can I broadcast the modifications to all other divices
INSTANTLY?