Hi Zimo!



 ZimoNitrome:

How do I instead create a function that gradually transform the input values to their softmax as p → inf?


There are lots of possibilities.  (You could choose between them by
imposing further conditions on the behavior of your function.)
Here is an illustration of one possible choice:
>>> import torch
>>> torch.__version__
'1.7.1'
>>> _ = torch.manual_seed (2021)
>>> def gradual (values, p):
...     return torch.where (values == values.max(), 1.0 - (1.0 - values)**p, values**p)
...
>>> values = torch.rand (5)
>>> values
tensor([0.1304, 0.5134, 0.7426, 0.7159, 0.5705])
>>> gradual (values, 1)
tensor([0.1304, 0.5134, 0.7426, 0.7159, 0.5705])
>>> gradual (values, 2)
tensor([0.0170, 0.2636, 0.9337, 0.5125, 0.3254])
>>> gradual (values, 3)
tensor([0.0022, 0.1353, 0.9829, 0.3669, 0.1857])
>>> gradual (values, 5)
tensor([3.7761e-05, 3.5664e-02, 9.9887e-01, 1.8803e-01, 6.0419e-02])
>>> gradual (values, 10)
tensor([1.4259e-09, 1.2719e-03, 1.0000e+00, 3.5355e-02, 3.6504e-03])

Best.
K. Frank