Hi @ptrblck ,
thanks for the answer.
The documentation sais:

Currently SyncBatchNorm only supports DistributedDataParallel with single GPU per process.

This “single GPU” kind of irritates me. What does it mean?
I am also asking because detectron2 still uses “FrozenBatchNorm2d”: https://github.com/facebookresearch/detectron2/blob/master/detectron2/modeling/backbone/resnet.py#L50 10