Add input tensor to the optimizer:

input.requires_grad = True
optimizer = optim.Adam([input], learning_rate)

Note that Variables don’t exist anymore in the newest versions on PyTorch.
In case, if you want to add input tensor along with model parameters to optimizer:
params_to_train = [input]
params_to_train += list(model.parameters())
optimizer = optim.Adam(params_to_train, learning_rate)


Yes,  adding the input to the optimizer will make input trainable. Make sure that grad of input tensor is turned on.

This may help you:

  
    
    
    Add one tensor from dataloader to optimiser 
  
  
    I want to optimize my init_tensor instead of optimizing the model. My data loader train_dataloader gives the initial value of init_tensor. 
model.eval()
for inputs in train_dataloader:
     init_tensor = inputs["init_tensor"]
     optimizer = optim.SGD(init_tensor, learning_rate)
     output = model(init_tensor)
     loss = loss_fn(output)
     model.zero_grad()
     loss.backward()
     optimizer.step()



Defining the optimizer inside the training loop doesn’t look right. Will this hinder opti…