I mean that I found the source of the problem, using topk with k=0 makes the gpu “unusable” for the next operation,
example
x = torch.Tensor([1,2,3,4,5,2,1,-1,-4,12,4]).cuda()
torch.topk(x,k=0,largest=False)

and then running any command that utilizes gpu (I ran it on a different cell)
x = x**2

will result in an error,
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-d4dbc0d35a05> in <module>()
----> 1 x = x**2

RuntimeError: CUDA error: device-side assert triggered

When I put it in a script and run it with CuDA_LAUNCH_BLOCKING I get the following stack trace (for the same snippet of code as above)
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [0,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [1,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [2,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [3,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [4,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [5,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [6,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [7,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [8,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [9,0,0] Assertion `writeIndex < outputSliceSize` failed.
/pytorch/aten/src/THC/THCTensorTopK.cuh:107: gatherTopK: block: [0,0,0], thread: [10,0,0] Assertion `writeIndex < outputSliceSize` failed.
THCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCTensorTopK.cu line=188 error=710 : device-side assert triggered
Traceback (most recent call last):
  File "script.py", line 3, in <module>
    torch.topk(x,k=0,largest=False)
RuntimeError: cuda runtime error (710) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCTensorTopK.cu:188