Hey @Sergius_Liu
Here is an example: https://pytorch.org/docs/stable/notes/ddp.html 75

I am not very sure what "single process per device " mean…

There are two modes in DDP: Single-Process Multi-Device mode (this is retiring) and Single-Process Single-Device mode (this is recommended). Please see this issue 34 for more context, and let us know if you have concerns.