Hello!
First of all, I appreciate your trying to help me out 
But I think you haven’t read the OP’s question carefully, which was “adding auxiliary branches to pretrained network”.
The problem with this is that pretrained models already have:

fixed architecture, which includes fixed size of weights as you’ve pointed out
fixed input arguments for forward()

Now, what I mean by adding auxiliary branches (and I’m quite confident OP also meant the same thing) is something like this:
A. Original module : out = module(input)
B. Modified module: out, aux_out = module(input)
I already know how to modify the intermediate module like B.
However, the problem is with 2. fixed input arguments for forward():
Even if I change the modified module to output a tuple (out, aux_out), the following module will not be able to accept that as an input.
The workaround may be using forward hooks to directly access the intermediate outputs of hidden layers, but I’m not so sure how the gradient will be affected in this case.