I saw you had a pointer to the hogwild training example. Could you please elaborate your use case? Are you looking for distributed data parallel training (like nn.parallel.DistributedDataParallel) or specifically asking for hogwild?