Solved by rasbt in post #2 
              
              
                Parameter is basically something extra in addition to Variable. E.g., if you want to optimize the tensor values also, then you would use Parameter. 
E.g., let’s say you define a model via nn.Module 
class ResNet(nn.Module):
    ...

And then pass it to the optimizer: 
...
model = resnet18(num_classe…