Solved by ptrblck in post #2 
              
              
                Your assumption seems to be correct based on this small experiment: 
N, H, W = 16, 224, 224

# increasing groups
norm = nn.GroupNorm(num_groups=3, num_channels=6)
x = torch.cat((
    torch.randn(N, 2, H, W) * 2 + 5,
    torch.randn(N, 2, H, W) * 4 + 10,
    torch.randn(N, 2, H, W) * 6 + 15,
), dim=1…