Hi,
sorry I did a mistake, but its now corrected…so x was out. I don’t even know if this loss function would work tbh, because gamma is not required to be in range [0, 1] and could just shoot toward +/- inf.
For the second solution I don’t know how to correctly implement it myself…so you would write something like:
gamma = nn.Parameter(torch.tensor([.5]))
optim = optim.Adam(itertools.chain(model.parameters(), gamma), lr=1e-3)

but if you run this, it will give you an error, because iterating over gamma returns you a non-leaf tensor, see:
for g in gamma:
    print(g.is_leaf)

Hopefully someone can point out my error