You are most likely hitting this issue 371, which is already fixed in the nightly CUDA11 pip wheels (and was not an issue in other binary configs), so you could either update to the nightly pip wheel, or use the CUDA10.2 or conda binaries.