Hi Bin!



 111414:

if there exists an efficient way to change the dimension-reduction function from inner product to the operation I indicated (e.g. L2 norm of residual)?
def g(a, b):
    return (a - b).norm()



I think, in general, if your g() is built from a broadcastable operation
and a reduction operation, you can accomplish your goal with pytorch
broadcasting.


I made some efforts but my solutions are all with inefficient nested for-loops.


In your specific case, you can use unsqueeze() in the appropriate
locations to replace the nested loops with  broadcasting, and then
apply norm():
>>> torch.__version__
'1.9.0'
>>>
>>> _ = torch.manual_seed (2021)
>>>
>>> m, n, p = 3, 4, 5
>>>
>>> A = torch.rand(m, n)
>>> B = torch.rand(n, p)
>>>
>>> def g(a, b):
...     return (a - b).norm()
...
>>> C = torch.empty(m, p)
>>> for i in range(m):
...     for j in range(p):
...         C[i, j] = g(A[i], B[:, j])
...
>>> C
tensor([[1.0094, 0.6486, 0.8481, 0.7942, 0.4473],
        [1.3406, 1.1867, 0.9334, 0.7280, 0.8513],
        [0.5945, 0.7856, 0.9520, 1.2410, 0.7428]])
>>>
>>> (A.unsqueeze (-1) - B.unsqueeze (0)).norm (dim = 1)
tensor([[1.0094, 0.6486, 0.8481, 0.7942, 0.4473],
        [1.3406, 1.1867, 0.9334, 0.7280, 0.8513],
        [0.5945, 0.7856, 0.9520, 1.2410, 0.7428]])

Best.
K. Frank