RuntimeError: CUDA out of memory. Tried to allocate 2.83 GiB (GPU 6; 31.75 GiB total capacity; 28.25 GiB already allocated; 2.09 GiB free; 28.60 GiB reserved in total by PyTorch)
Some errors happend, so GPU 6 is empty.