Are you using the same vocab size and embedding dimensions for both use cases (pretrained and trained from scratch)?
If the dimensions are the same, the code should not create an OOM error in one use case while the other is working fine.