Thanks for posting @sunshichen Could it be the case that only some process get OOM while other process still not and it just get hangs because it’s waiting for dist.barrier()? Or you are observing all processes get OOM and all hangs?
It would also be good if you can have a self contained small script to repro the issue so that we can help you debug it.