For anyone interested, I wrote a blog post explaining the adaptive softmax, with a Pytorch implementation: https://towardsdatascience.com/speed-up-your-deep-learning-language-model-up-to-1000-with-the-adaptive-softmax-part-1-e7cc1f89fcc9 158