What if we don’t allocate new ndarrays/tensors in the inner loop?
c = torch.zeros_like(a)
%timeit torch.mul(b, a, out=c)  # 2.23 µs ± 224 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

cn = np.zeros_like(an)
%timeit np.multiply(bn, an, out=cn)  # 866 ns ± 36.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Allocation can’t be the main source of PyTorch overhead either. It still isn’t as fast as naive NumPy.
For reference, my benchmarks were performed on a 2-core MacBook Pro, with Python 3.7.4 installed via Homebrew, numpy 1.17.2 installed via pip (and thus built vs openblas), and torch 1.3.1 installed via pip.