CUDA_LAUNCH_BLOCKING=1 will synchronize each call and thus slow down your code.
Based on your description and the output of nvidia-smi your devices might indeed be overheating.
Could you open the case and put a large fan in front of the machine while it’s running?
If that’s possible, check nvidia-smi -l and observe if the temperature would still be rising to critical levels.