I’m not sure, if nn.GroupNorm needs less memory than nn.BatchNorm in the layer implementation.
But in general you save memory, because you can lower the batch size up to 2 without sacrificing much accuracy.
You can see this effect in Figure1 in the paper 6.