I would assume PyTorch might have a larger overhead, but I’m getting similar numbers for your test:
%timeit "import numpy as np; test = np.random.rand(3,3)" "test + 1"
> 6.31 ns ± 0.188 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

%timeit "import torch; test = torch.rand((3, 3))" "test + 1"
> 6.12 ns ± 0.0184 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

for PyTorch 1.5.0.dev20200122.