You can use the groups parameter in nn.Conv2d to achieve this. Try to set groups=in_channels and each input channel will have its own set of filters.
Note that both in_channels and out_channels have to be divisible by groups.