Solved by ptrblck in post #4 
              
              
                It seems you still passing input to the layer and apply the alpha parameter later or could you explain how you are passing alpha to e.g. the linear layer? 

PyTorch applies broadcasting, so if alpha is a scalar tensor you could directly run the posted line of code. 
On the other hand, even if alpha …