pytorch will use all the cores on a single machine.
If you want to use 25 Xeon machines, then you will have to write some special logic using our torch.distributed functions: http://pytorch.org/docs/master/distributed.html 98