To illustrate, this would be something like the pseudo model I would like to create.
class Combinator(nn.Module):
    def __init__(self, modelA, modelB):
        super(Combinator, self).__init__()
        #Linear coefficient vector
        #[Wa, Wb]
        self.linweights = nn.Parameter(torch.ones(2))
        
        #Combined model
        #self.model = Wa*modelA + Wb*modelB
             
    def forward(self, x):
        x = self.model(x)
        return x

The problem I have is with creating that resulting model combining the others and if it makes sense to, for example, setting the models parameters to “requires_grad = False” and wanting only the coefficients I’ve named “linweights” to be trainable and changed by the backprop.