Solved by kyungyunlee in post #4 
              
              
                Hmm…I am very new to pytorch so I am still experimenting as well, so I am not 100% sure and correct me please if I am wrong  
But first, the output of criterion/loss function is autograd Variable, since it can be used like 

loss = criterion(…) 
loss.backward() 

And looking at the scheduler class,…