Hi Aerys!



 AerysS:

How can I use that sparse softmax as a functional one (that is, it is not a layer)?


I didn’t look at the github code you linked to, but, in general, if you
have a (properly-written) torch.nn.Module or
torch.autograd.Function, you can instantiate it “on the fly” and
call it, rather than using it as a “layer” in a model.
Thus, for example:
probs = torch.nn.Softmax (dim = 1) (logits)

is the practical equivalent of:
probs = torch.nn.functional.softmax (logits, dim = 1)
# or
probs = torch.softmax (logits, dim = 1)

Best.
K. Frank