Solved by shivammehta007 in post #4 
              
              
                I guess the cleanest method would be to use a sigmoid activation on w and then multiply it 
w = torch.randn(3,3)
w = torch.sigmoid(w)

output = w * y_pred1 + (1-w) * pred_2

Sigmoid has a very smooth gradient and is differentiable everywhere. 
[Sigmoid-function-2]