Hey @adarsh-kr,
There is a destroy_process_group API to clear the default ProcessGroup
instance. If you would like to create multiple ProcessGroup instances, you can do so using the new_grou 24 API.

  
      github.com
  
  
    pytorch/pytorch/blob/05f00532f52883d29a08d96b2961042cc41573ab/torch/distributed/distributed_c10d.py#L530-L577 32

def destroy_process_group(group=group.WORLD):
    """
    Destroy a given process group, and deinitialize the distributed package

    Arguments:
        group (ProcessGroup, optional): The process group to be destroyed, if
                                        group.WORLD is given, all process
                                        groups including the default one will
                                        be destroyed.
    """
    global _pg_map
    global _pg_names
    global _pg_group_ranks
    global _default_pg
    global _default_pg_init_method
    global _group_count

    if group == GroupMember.NON_GROUP_MEMBER:
        return



  This file has been truncated. show original