Your assumption is right! Your layers both have two filters with a different number of channels.
conv1 = nn.Conv2d(1, 2, 3, 1, 1)
print(conv1.weight.shape)
> torch.Size([2, 1, 3, 3])
conv2 = nn.Conv2d(2, 2, 3, 1, 1)
print(conv2.weight.shape)
> torch.Size([2, 2, 3, 3])

The filter shape is defined as [nb_filters, in_channels, h, w].
So besides the changing number of input_channels, we still have two filters.