Hi Yaroslav!



 boomland:

I tried to write my custom layer for sigmoid. But it doesn’t work.


Your SigmoidCustom works for me:
>>> import torch
>>> torch.__version__
'1.9.0'
>>> _ = torch.manual_seed (2022)
>>> class SigmoidCustom(torch.nn.Module):
...     def __init__(self):
...         super(SigmoidCustom, self).__init__()
...
...     def forward(self, x):
...         return 1. / (1 + torch.exp(-x))
...
>>> s = torch.randn (5)
>>> t = s.clone()
>>> s.requires_grad = True
>>> t.requires_grad = True
>>> s.sigmoid().sum().backward()
>>> s.grad
tensor([0.2477, 0.2433, 0.2467, 0.2061, 0.2474])
>>> SigmoidCustom() (t).sum().backward()
>>> t.grad
tensor([0.2477, 0.2433, 0.2467, 0.2061, 0.2474])
>>> torch.allclose (s.grad, t.grad)
True

Best.
K. Frank