Ho right, you can only step in the cpu code. But it can run cuda code fine even though you cannot break into it.
I did not knew you could put breakpoints directly into the kernels. But the fact that you use python should not be a problem. nvvp works quite fine to inspect python code.