yeah just filed a issue about this, we don’t have a destructor or API that could call to release those ports now, tracking it here [c10d] destruction of Store objects · Issue #72025 · pytorch/pytorch · GitHub