nn.DataParallel will scatter the model to all devices and gather their outputs on the master device. This blog post 33 gives a good overview.
Since you are assigning a new value to running, I think the best way would be to have a look how data parallel works internally 16 and adapt these methods to your use case.