Actually, my goal is not to split on 2 GPUs. I tested my network with groups=2 and groups=1, and wondering if I loose some information when applying groups=2, because printing the state dict (if I do it correctly) shows in the second layer only half of the channels from the first layer.
That means with groups=2:

layer output channels : 96   --> 2.layer input channels : 48   (see first post)

But with groups=1:

layer output channels : 96   --> 2.layer input channels : 96   (as expected)

I print the state dict this way:
    for param_tensor in net.state_dict():
      print(param_tensor, "\t", net.state_dict()[param_tensor].size())

So the main question from my perspective is, if I loose some information when using groups=2 ?