If your model’s parameters are on the GPU the step() operation would also use the GPU to update the parameters. I don’t know which iteration you are profiling, but guess it might be the first one and you might see the initialization of some buffers here. The update can be seen here.