When I do profile on my train code, I found that there would be aten::copy_ and aten::to operations before  optimizer.step().So I wonder if  optimizer.step()  is calculated on cpus.

pfofilr1206×391 14.9 KB