The short answer is: Variables are deprecated, nn.Parameters require gradients and are automatically registered inside nn.Modules (so that they are returned in their state_dict and pushed to the device via model.to()), tensors are just array-like objects which can require gradients.