gcramer23:

Hi why won’t the described behavior be seen in 3.


i think it has something to do with  the kernels.
in my ‘real’ code, x is computed using some forward operations, kernels could take way more time to get them + the realtime load on the gpu at that moment. in the snippet code, i use torch.rand, i assume it is an easy task that we can get the results fast enough that subsequent operations wouldnt be blocked.



 gcramer23:

It seems as though this a python problem with the if statement because of the CPU.


yes, the condition is evalauted on gpu and results in a boolean tensor. but in order for the cpu to access it, it ahs to transfer it to cpu.



 gcramer23:

I am unsure if this will help since I can’t reproduce the problem, but I will torch.equal work for 1?


i tried almost every imaginable way to get the value of the cuda boolean tensor without causing the blocking but it didnt work. this include torch.equal, indexing, evaluating the condition beforehand, numel(), … all this results in a blocking (synchronization) in order to make sure the cpu evaluates the right value of the tensor. if it is not the case, and the access to the tensor is not synchronized, cpu will produce wrong results.
it doesnt seem there is a way to escape this. lazy transfer from gpu-2-cpu is unsafe and leads to wrong results as mentioned in header parag.