Hello @alexis-jacq,
I have just tried to implement your module, but it doesn’t work properly.
I get an an error:

image.png1003×507 70 KB

So, i removed your SigmoidBias module:
class SigmoidBias(nn.Module):
    def __init__(self, output_features, bias=True, inplace=False):
        super(SigmoidBias, self).__init__()
        self.inplace = inplace
        if bias:
            self.bias = nn.Parameter(torch.Tensor(output_features))
        else:
            self.register_parameter('bias', None)
        if self.bias is not None:
            self.bias.data.uniform_(-0.1, 0.1)

    def forward(self, input):
        output = torch.sigmoid(input)
        if self.bias is not None:
            output += self.bias.unsqueeze(0).expand_as(output)
        return output

and it worked fine.
Could you help me to beat the problem?