Ah, thanks alot.
If someone comes looking into this thread, the intuition is that if we define the convolutional layer as:
nn.Conv2d(3, 16, 5, padding=1),
we define 16 filters, which all have the dimension (5x5x3). I.e, the filters look at the entire depth of the input. The output would then be the output from every filter, stacked on top of each other.
Refer to the excellent link in ptrblck’s answer for further explanation.