Ahmed_Abdelaziz:

Why a tensor with requires_grad will not get gradients while the Parameter will do?


Both will get gradients, but in your initial post you didn’t set requires_grad=True in the tensor creation, so self.embeddings wouldn’t get any gradients.
Additionally to that, a registered nn.Parameter will be automatically pushed to the device, if you call model.to() and will also be returned in model.parameters(), while a tensor will not.