Sorry, not a solution, but may I ask what your task is? Classification? Translation? 120k is quite a large vocabulary. Training an RNN is tricky enough, so always try to minimize the vocabulary as much as possible to make it easier for the network. Some common considerations/steps:

What’s the distribution of words? I would hazard a guess that the top 10k words might already cover 95% of your content. For classification, that’s probably already enough.
What do you do for preprocessing? I often work with informal text. While I’m fine with “lol”, I don’t want it to happen that “loool” or “loooool” count as different words. Such noise can blow up a vocabulary quickly.
Do you have many named entities (i.e., names of persons, locations, organizations, etc.) or numbers in your text? In case of translation, such “special word” won’t get translated anyway just copied. So a common approach is to first replace such words with a special token (e.g., “I met Alice in London” to "I met  in ") do the translation and then add the names back in.

Just some food for thought.