Two ways I’d see doing it:
First to have two separate embeddings.  One embedding learns, the other uses pre-trained weights.  Select the embedding to use depending on the value of the input.
The other approach would be to overwrite the pretrained parts of the embedding at the beginning of each batch to undo the results of the previous optimizer step.