Hello, I am trying to run my model on an Xavier NX. It seems to take about 17 seconds to copy a torch tensor of size (320 x 480), dtype: torch.uint8 to the host memory. Any suggestion to deal with this bottleneck would be helpful. Code snippet:
 with torch.no_grad():
            a = model(img)    # a['out'] >> torch.float32; size: [1 x 1 x 320 x 480]; no Grad; on CUDA
result_gpu = (a['out'][0][0] > 0.1).type(torch.uint8) # result_gpu >> torch.uint8; size: [320 x 480]; no Grad; on CUDA
start = time.time()
result_cpu = result_gpu.cpu()
end = time.time() 
print(end - start)   # 17.0668951