I’m also learning from this tutorial. One question is does the simplest case running on GPU or CPU? I tried to use nvprof to checkout, but no kernels detected, so it spawns 2 processes on CPU, am I right? How to make it to run on 2 GPU processes? I tried to create the tensor on CUDA device like this:
tensor = torch.zeros(1).cuda(rank). But still nvprof didn’t find any kernels launched, am I missing something?