Hmmm so if correctly understood, you have a tensor A, which is computed once per epoch. You need A inside your model but you don’t want it to be computed in parallel.
I find a simple way of doing it
Depending on the role they play in your network you can wrap your network keeping that one as external like:
class wraped_net(nn.module)
__init__(A,model)
self.A = A
self.my_real_model = nn.Datapralel(model)
forward(inputs):
return self.my_real_model(inputs,A)

Drawback of this approach is that you will be copying A in each gpu.
If you use it in a very simple way, you can externalize that operation at the begining
class wraped_net(nn.module)
__init__(A,model)
self.A = A
self.my_real_model = nn.Datapralel(model)
forward(inputs):
do_something_with_A_output=do_something_with_A(inputs,A)
return self.my_real_model(do_something_with_A_output)