I would remove the weight_decay argument to Adam and explicitly add the L2 penalty to the loss
for p in model.parameters():
    if p is not None:
        loss += options['reg'] * p
loss.backward()
torch.nn.utils.clip_grad_norm(model.parameters(),options['clip_gradient_norm'])
optimizer.step()