Your current code raises another error, so that I wasn’t able to reproduce the cublas issue with it:
RuntimeError: index_put_ with accumulate=False does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'. You can turn off determinism just for this operation if that's acceptable for your application. You can also file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation.

However, this simple code raises the same error message:
import torch

torch.use_deterministic_algorithms(True)
x = torch.randn(1024, 1024, device='cuda')
y = torch.matmul(x, x)

RuntimeError: Deterministic behavior was enabled with either `torch.use_deterministic_algorithms(True)` or `at::Context::setDeterministicAlgorithms(true)`, but this operation is not deterministic because it uses CuBLAS and you have CUDA >= 10.2. To enable deterministic behavior in this case, you must set an environment variable before running your PyTorch application: CUBLAS_WORKSPACE_CONFIG=:4096:8 or CUBLAS_WORKSPACE_CONFIG=:16:8. For more information, go to https://docs.nvidia.com/cuda/cublas/index.html#cublasApi_reproducibility

and works fine with the env variable:
CUBLAS_WORKSPACE_CONFIG=:16:8 python script.py 
/opt/pytorch/pytorch/torch/__init__.py:470: UserWarning: torch.use_deterministic_algorithms is in beta, and its design and functionality may change in the future. (Triggered internally at  ../aten/src/ATen/Context.cpp:69.)
  _C._set_deterministic_algorithms(mode)