Well, you answered your own question. You can freeze layers in PyTorch by setting requires_grad=False to a layer’s parameters. They will not be updated during training. You can then load the model, swap out the weights of the embedding layer with other learnt weights and save the model again (In transformers you can use model.save_pretrained()).
I am not sure how much help you need. If you need a step-by-step guide, I fear I do not have the time to help with that. The above should you help you a bit.