I have reviewed the forums and docs about adding a trainable parameter to the model, by defining trainable_attr = nn.Parameter(None, requires_grad = True) and after that register it to the target model model.register_parameter(name, params). But How can I train a parameter outside of the model, for instance, I have a custom loss which is the addition of two losses, e.g., MSE and Cross_Entropy:
total_loss = mse_loss *gamma + cross_entroppy_loss *(1-gamma)
How can I define and train gamma?