I see! That requires modifying the forward method of  modelA?
Here is a few words about what I am trying to do.
I have ModelA :
[input/image -> encoder1 -> header1 -> output1].
The model was trained and has pretty good accuracy.
Now I want to add a 2nd task.
For that purpose, I run the entire dataset through ModelA and for each example, save the  activation at the last layer of the encoder as numpy array file.
Those numpy arrays are then used as the inputs to modelB  [input2 -> header2 -> output2]. Then run optimization and save the weights of modelB.
Finally, I want to combine modelA and modelB into a combo-model with 2 heads:
combo_model:  [input/image -> encoder1 -> header1 -> output1 
                                      \-> header2 -> output2

This combo_model is only used at test time.
(I understand I could start with the combo model architecture and train the modelA part with freezing modelB branch and then optimize modelB with freezing ModelA. IMHO, the approach is more cumbersome!)