Given that the Embedding’s forward is literally just
return F.embedding(
            input, self.weight, self.padding_idx, self.max_norm,
            self.norm_type, self.scale_grad_by_freq, self.sparse)

you could do this in your forward yourself and pass in the torch.cated weights.
You might even just store the weight as a parameter (or even a buffer) instead of having its own Embedding layer. (Or subclass Embedding to EmbedddingWithFrozen that does all this under the hood.)
There is a slight drawback that you’ll be copying the embedding quite a bit. With more trickery, you could look into reducing that by doing the embedding in two stages, but it is considerably less straightforward.
Some unsolicited style advice:



 Sirui_Li:

        self.freeze_rel_embeddings.weight.data = self.config.freeze_init_rel_embs
        self.freeze_rel_embeddings.weight.requires_grad = False
        self.trainable_rel_embeddings.weight.data = self.config.init_rel_embs



I would probably write this as
        with torch.no_grad():
            self.freeze_rel_embeddings.weight.copy_(self.config.freeze_init_rel_embs)
        self.freeze_rel_embeddings.weight.requires_grad_(False)

Best regards
Thomas