albanD:

torch.cuda.synchronize()


Thanks for the tip, I did not know.
Stacking slows down the code but slicing too:
A = torch.randn(20000).to(device)
%timeit AA = A
%timeit AA = A[:-1]
%timeit A[:-1] = A[:-1]

18.4 ns ± 3.15 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
1.7 µs ± 3.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
14.6 µs ± 2.1 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Is that the best I can do?
Moreover, I discovered that there is now complex value related functions with PyTorch!
https://pytorch.org/docs/stable/torch.html 20
We find torch.angle(), torch.real() and torch.imag().
So I upgraded PyTorch to the latest (1.5) on my calculation server and ran the usage example from the official website for the torch.angle() function:
torch.angle(torch.tensor([-1 + 1j, -2 + 2j, 3 - 3j]))*180/3.14159

and got:
RuntimeError: Could not infer dtype of complex

Instead of the results from PyTorch documentation:
tensor([ 135.,  135,  -45])

If there is still no official complex tensor support, how one can use those functions?