I don’t know which pip wheel you’ve installed, but you could try to rebuild PyTorch with the same cublas version and check, if you might be seeing an already fixed issue.
Also, make sure that you are not running out of memory, as cublas might raise this unhelpful error message if it’s unable to allocate memory internally.
In any case, I would also recommend to stick to the latest PyTorch release (1.8.1), as it ships with the latest bug fixes (you might also hit an internal PyTorch issue, which might have been fixed after the 1.6.0 release).