Solved by ptrblck in post #2 
              
              
                That should be possible, if you create your input using requires_grad=True and pass it to the optimizer. 
Have a look at this cosine similarity example I’ve written yesterday.