agt:

But I’m not only passing my input into pytorch layers.


It seems you still passing input to the layer and apply the alpha parameter later or could you explain how you are passing alpha to e.g. the linear layer?



 agt:

In that case, would I have to manually loop over the input batch and build the new tensor after doing my custom computations on each item of the batch?


PyTorch applies broadcasting, so if alpha is a scalar tensor you could directly run the posted line of code.
On the other hand, even if alpha has the shape [batch_size] it should still work (and you might need to unsqueeze() dimensions to enable broadcasting, but it depends on the shapes of the other tensors).