You can use narrow to your advantage:
a = torch.randn(5, 2048, 5)
print(((a[:,1:,:] - a[:,:-1,:])-(a.narrow(1, 1, a.size(1)-1)-a.narrow(1, 0, a.size(1)-1))).abs().max().item())
%timeit a[:,1:,:] - a[:,:-1,:]
%timeit a.narrow(1, 1, a.size(1)-1)-a.narrow(1, 0, a.size(1)-1)

gets me
0.0
14.3 µs ± 190 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
11.9 µs ± 15.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

so by going to PyTorch directly I even shave off a couple of µs.
Best regards
Thomas