Variable is used for all sorts of values. Model weights, RNN hidden states, and input data. Most of these do not need gradients.
Declaring a value using nn.Parameter tells PyTorch two things. 1. that gradients are needed. 2. that the optimiser must optimise these values.