Solved by akernel in post #2 
              
              
                You could write your own nn.Module that includes a trainable alpha parameter: 
class MyLayer(nn.Module):
    def __init__(self, *args, **kwargs):
        self.alpha = nn.Parameter(torch.tensor(0.))
        # Other necessary setup
        super().__init__(*args, **kwargs)

    def forward(self, x):
 …