Just as an update, if I changed the line for random number generated to be on cuda:
idx = torch.randint(0, 512, (1,), device='cuda')

then the long kernel runs as quickly as the short kernel (like 5e-5s).  I believe this is the correct thing to do here, but I’m still not comfortable with the change of behavior between long and short kernels in the original code.
Also, I tried with torch version 1.8.1, and even putting the randint line on cuda resulted in 1.4s for the long kernel.  Can someone from the PyTorch team confirm this is?  Is there a workaround if for some reason I have to stick with 1.8.1?  Thanks.