Hi Ptrblck! I got the same problem that the parameters are not updating. I am using torch.nn.Parameters as you suggested, but still does not work. At the last line of init, I created the model parameter named combine_weight in order to obtain trainable weights.
class RNN(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, num_classes):
        super(RNN, self).__init__()
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.GRU = nn.GRU(input_size, hidden_size, num_layers, batch_first=True)
        self.fc = nn.Linear(hidden_size, num_classes)
        self.combine_weight = nn.Parameter(torch.ones(12) / 12, requires_grad=True).to(device)

And I checked whether the combine_weight has existed in model.parameters and optimizer.param_groups. And yes the print results show that the parameter does exist.
down_model_optimizer = get_optimizer(config, downstream_model, given_learning_rate=config.PER_learning_rate)
i_list = [down_model_optimizer.param_groups[0]['params']]
print(i_list)

print results >>>[[Parameter containing:
tensor([0.0833, 0.0833, 0.0833, 0.0833, 0.0833, 0.0833, 0.0833, 0.0833, 0.0833,
        0.0833, 0.0833, 0.0833], requires_grad=True)

Although this created parameter is in model and optimizer, the number is still 0.0833 without updating.
what other places should I also check? Do you think what may cause this problem? Thank you so much!