I searched the internet for some references, but I only found this as an example pytorch implemention. Maybe you can cross check your code with this. https://github.com/leimao/Two_Layer_Hierarchical_Softmax_PyTorch/blob/master/utils.py#L98 137