You don’t have to specify the range name in range_pop, but besides that it should work.
My workflow would be:
# Setup
...

# enable the profiling, such that nvprof or nsys doesn't capture operations before this point
torch.cuda.cudart().cudaProfilerStart()

torch.cuda.nvtx.range_push("forward")
output = model(input)
torch.cuda.nvtx.range_pop()

# don't profile this
optimizer.zero_grad()
loss = criterion(output, target)

torch.cuda.nvtx.range_push("backward")
loss.backward()
torch.cuda.nvtx.range_pop()

torch.cuda.nvtx.range_push("optimizer.step()")
optimizer.step()
torch.cuda.nvtx.range_pop()

torch.cuda.cudart().cudaProfilerStop()

You can of course use nested ranges, if needed.
Then run the script via:
# with nsys
nsys profile -w true -t cuda,nvtx,osrt,cudnn,cublas -s cpu -o my_report --capture-range=cudaProfilerApi --stop-on-range-end=true --cudabacktrace=true --cudabacktrace-threshold=10000 --osrt-threshold=10000 -x true python script.py

# or with nvprof
# single-process
nvprof --profile-from-start off -fo %p.nvprof python script.py

# multi-process
nvprof --profile-child-processes --profile-from-start off -fo %p.nvprof python -m torch.distributed.launch  --nproc_per_node=2 script.py

Note that my nsys cmd creates stack traces etc., so you might want to slim it down a bit.