You may also be interested in this implementation:

  
      
      GitHub
  
  
    

facebookresearch/adaptive-softmax 384

Implements an efficient softmax approximation as described in the paper "Efficient softmax approximation for GPUs" (http://arxiv.org/abs/1609.04309) - facebookresearch/adaptive-softmax


  
  
    
    
  
  


Giving very good results for the LM task.