How is the gradient for torch.nn.Embedding calculated? The weight is simply a lookup table - is the gradient being propagated only for the certain indices?
I also have a side question if anyone is knows anything about fine-tuning the BERT model. Are the embedding layers weights adjusted when fine-tuning? I assume they are since the paper states:

… all of the parameters are fine-tuned using labeled data from the
downstream tasks.

But wouldn’t changing the embedding parameters (for example the positional encoding) distort the model? I am not sure.