It seems the problem isn’t anything specific to the ops in the example above, but rather number of iterations. I get the same effect with the following code:
a = torch.ones([10000, 10000]).cuda()
    
with torch.autograd.profiler.profile(use_cuda=True) as pr:
    for i in range(400):
        b = a * 10.0

t = pr.table(sort_by='cpu_time_total')
print(t)  

This is the printout table from rows 120 to 130 or so. Gives the same sudden jump in CPU time, which then continues to climb slowly until it’s significantly greater than the CUDA time.

Name            CPU time        CUDA time            Calls        CPU total       CUDA total
mul             28.688us       8523.766us                1         28.688us       8523.766us
mul             29.667us       8509.441us                1         29.667us       8509.441us
mul             77.528us       8491.265us                1         77.528us       8491.265us
mul            726.486us       8133.500us                1        726.486us       8133.500us
mul           8383.802us       8525.750us                1       8383.802us       8525.750us
mul           8387.009us       8410.250us                1       8387.009us       8410.250us
mul           8401.606us       8426.500us                1       8401.606us       8426.500us
mul           8404.111us       8478.750us                1       8404.111us       8478.750us

Not sure if the op is actually being done on the CPU, or if this is an anomaly with the profiler…