Operations that require a synchronize will block (see cudaStreamSynchronize and cudaDeviceSynchronize). In particular, device-to-host transfers require a synchronize, which is why print will block.
Tensors are backed by a python storage that holds a pointer to data that can be on GPU or CPU.