I think the issue I mentioned was fixed recently in pytorch, I have installed the latest PyTorch version from the nightly build: ‘1.11.0.dev20211130+cu102’, and I do not get the following error anymore:
RuntimeError: linearIndex.numel()sliceSize nElemBefore == value.numel()INTERNAL ASSERT FAILED at “/pytorch/aten/src/ATen/native/cuda/Indexing.cu”:250, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor10231
Now I can set torch.use_deterministic_algorithms(True) in my code, but the results are still not reproducible.
Thanks!