Hello! Sorry to write again, but I haven’t been able to advance much, I was wondering if you may have an idea about what to do.
So far, the only way I’ve found to multiply what I call the “coefficients” to train is by creating a nn.Parameter for each model I want to combine (say, self.linwA = nn.Parameter(abs(torch.randn(1)),requires_grad=True) ) and multiply it by each of the parameters of the model, something like this:
def mulpar(self, coefficient, model):    
    for k, v in model.state_dict().items():
        model.state_dict()[k][:] = coefficient.expand_as(v)*model.state_dict()[k][:]
    return model

However, I think it doesn’t work, because the resulting model is no longer trainable (requires_grad = False) and I think that’s causing a “RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn” error later when I want to calculate the loss, there’s nothing to backpropagate.
Any pointers or ideas?