The dimensions are the same, however the pretrained embedding vocab size is  ~4x larger (400,000 vs 100,000). Does Pytorch update the entire embedding at once? Otherwise I don’t see how having a larger embedding could be an issue (a side from having more memory consumed, but what I was seeing seemed more like an explosion in memory usage).
Note: I messed around with it some more and suspect it is a bug in the vocab’s code. If the vocab size isn’t explicitly state (Instead of a size you can pass -1 and it acts as if you were getting the last index of your array) that’s when the issue occurs.