Solved by benoriol in post #7 
              
              
                Instead of doing somthing like: 
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) 
try 
optimizer = torch.optim.Adam(inputTensor, lr=learning_rate) 
And maybe, as you said, you should make sure requires_grad=True for the input tensor.