Hi Team,
Just out of curiosity, for torch.nn.Embedding, if we set padding_idx=0, will the first entry of embedding table be updated during model training(cause its special entry just for alignment and dry-run in models and would be unnecessary if we really want to make all weights of embedding table trainable).
Regards,
Zan