Sampled softmax loss
Hi, Does sampled softmax loss exist in pytorch? I cound not find it. Thanks
beegii
smth
beegii
We don’t have any such thing in the core. We’ll need to add them. Thanks for the pointer!
apaszke
ngimel
beegii
EDIT: sorry, I see that original link is to a page with a number of different softmax approximations, and NCE is one of them. I personally would be more interested in sampled softmax, as it tends to work better for me.
lopuhin
You may also be interested in this implementation: Giving very good results for the LM task.
vince62s
beegii
Any updates on this?? It probably isn’t a priority…but I secretly wish PyTorch has a big development team like TensorFlow and can add these functionalities easily!
windweller
smth
windweller