The inner step. As described below:
inner_loss = l2_loss(img_ano,prior)
inner_loss.backward()

img_ano.grad += out(img_ano.data()) # Changing the gradient by adding the output of network

img_ano = img_ano + step_size * img_ano.grad # (restore_optimizer.step()) Here I want to take a corresponding step with adam optimizer instead

loss = diceloss(img_ano,target_img)
loss.backward() # Gather network params gradients, ie. backpropagate the gradient step 
net_optimizer.step() # Update network params 

Thanks for the help!