The error is because your input is 3D (channels x h x w). You need a batch dimension (batch x channels x h x w), although it can be 1.
x = torch.nn.Conv2d(10, 10, [3, 3], groups=10) # in_channels = 10, out_channels = 10
img = torch.autograd.Variable( torch.randn(1, 10, 5, 5) )  # in_channels = 10
o = x(img)