Hi,
I tried as you say, but when I add the parameters to an optimizer:
params = [net.parameters(), loss.parameters()]
opt = optim.Adam(params, lr = 1e-3, weight_decay = 5e-4)

It invokes the error message:

File “/home/zhangzy/.local/lib/python3.5/site-packages/torch/optim/optimizer.py”, line 192, in add_param_group
"but one of the params is " + torch.typename(param))
TypeError: optimizer can only optimize Tensors, but one of the params is Module.parameters