Thanks Simon for you answer. I saw that it is possible to write the training code using the distributed package, also in the tutorial there is an example to do your own DistributedDataParallel for cpu. But I am doing the question, because in this tutorial is recommending to use the DistributedDataParallel provided in the framework, but far as I saw in the documentation, the current implementation only supports cluster with gpus. Is it true? Or I missing something?