Thanks @JuanFMontesinos. Cleaning it up, the final pipeline will look like this:
model.eval()
for batch_id, inputs in enumerate(train_dataloader):
     init_tensor = inputs["init_tensor"] # tensor to be optimized
     optimizer = optim.SGD(init_tensor, learning_rate)
     input_tensor = inputs["input_tensor"] # different images from loader
     input = init_tensor + input_tensor
     loss = 1
     while(loss > threshold):
          output = model(input)
          loss = loss_fn(output)
          model.zero_grad()
          optimizer.zero_grad()
          loss.backward()
          optimizer.step()

I still have a doubt,
How can we use the learned info of optimizer from inputs of batch_id=1 to improve the optimization of inputs of batch_id=2? As we are initializing optimizer for each inputs we are throwing out all it has learned from inputs of batch_id=1. So that the optimizer should spend almost same amount of time optimizing inputs with batch_id=2 as it did with inputs of batch_id=1. How can we improve this?