AndrewSoul:

I find the grad_m_o is different from grad_net, why?


How large is the difference?
I’m not sure, if the code you’ve posted is still the currently used one, but it seems to be that you should pass targets_layers as a list of strings.



 AndrewSoul:

Is the problem caused by not defined as nn.Module?


I think you are not using multiple GPUs, since you are never calling the model itself (and thus its forward method), but split the call into submodules and call them in a loop.
Could you try to call net(input) and check the GPU utilizations?