Two 2080TIs should work, so I think it might be a hardware issue.
However, it seems you’ve already changed a lot of parts of your system.
Regarding point 3 and 4, it seems you completely rebuilt your system.
Does your code only crash using the ImageNet example or also a very small model, e.g. a single linear layer and DataParallel?