Wouldn’t a simpler (and faster) implementation be:
class LinearWeightedAvg(torch.nn.Module):
  def __init__(self, n_neurons):
    super(LinearWeightedAvg, self).__init__()
    self.weight_inp1 = torch.nn.Parameter(torch.randn(1, n_neurons))
    self.weight_inp2 = torch.nn.Parameter(torch.randn(1, n_neurons))

  def forward(self, input1, input2):
    return input1 * self.weight_inp2 + input2 * self.weight_inp2

Also I would be careful with the weight initialization. Here I use randn for simplicity but something else should be used for sure.
Also the weight initialization of a Linear Layer depends on the sizes so might behave unexpectedly for very small numbers like you do.