Hi all,
I found something strange: pin_memory() and clone() somehow use a huge amount of CPU time.  I think tensor copy operation is somehow very inefficient for AMD CPUs.  If I force it to use less thread with OMP_NUM_THREADS=1 CPU usage comes down.
Here’s an example using clone().
import os, time, threading
import numpy as np
import torch

use_pin = True
run_on_separate_thread = False

X = torch.zeros(1).to('cuda')  # Initialize CUDA.

def run():
    S = torch.zeros(1).to('cuda')
    for _ in range(1000):
        A = np.zeros(4 * 1024 * 1024, dtype=np.float32)
        B = torch.from_numpy(A)
        if use_pin: B = B.pin_memory()
        B = B.to('cuda', non_blocking=True)
        S += B.sum()

    print('Sum = ', S)  # should be zero

usertime0, systime0, _, _, T0 = os.times()

if run_on_separate_thread:
    thr = threading.Thread(target=run)
    thr.start()
    thr.join()
else:
    run()

usertime1, systime1, _, _, T1 = os.times()

print('user %7.3f | sys %7.3f | real %7.3f' %
      (usertime1 - usertime0, systime1 - systime0, T1 - T0))

Interestingly, when I run the code in a separate thread somehow the problem becomes much worse (I ran each case three times):
If we call run() directly:
    Without pin_memory():
        user   3.520 | sys   0.280 | real   3.820
        user   3.580 | sys   0.220 | real   3.820
        user   3.520 | sys   0.320 | real   3.850
 
    With pin_memory():
        user  30.620 | sys   0.150 | real   3.880
        user  30.720 | sys   0.100 | real   3.890
        user  30.660 | sys   0.170 | real   3.890

    With pin_memory(), OMP_NUM_THREADS=1:
        user   4.050 | sys   0.020 | real   4.090
        user   4.040 | sys   0.020 | real   4.080
        user   4.070 | sys   0.020 | real   4.100

If we call run() on a separate thread:
    Without pin_memory():
        user   3.510 | sys   0.290 | real   3.820
        user   3.550 | sys   0.270 | real   3.840
        user   3.570 | sys   0.240 | real   3.820

    With pin_memory():
        user  40.080 | sys  27.930 | real   4.320
        user  41.980 | sys  31.910 | real   4.760
        user  43.390 | sys  27.080 | real   4.500

    With pin_memory(), OMP_NUM_THREADS=1:
        user   4.100 | sys   0.030 | real   4.140
        user   4.090 | sys   0.020 | real   4.130
        user   4.130 | sys   0.020 | real   4.170

I’m using Ryzen 1700X, GTX 1080, Python 3.7.4, and PyTorch 1.3.0 (also tried building PyTorch from latest source and got similar result).  Is this one of those cases where the underlying Intel MKL library doesn’t play nicely with AMD CPUs?  Or am I doing something wrong?
For now, I think I’ll just skip pin_memory() altogether (and be careful never to copy tensor from CPU to CPU), but it might be nice if a fix is available. In my code, it was using so much CPU that it became the bottleneck even though everything after that was running on CUDA.