I found this article helpful in detailing the underworkings of the Conv2d operation:

  
      

      Medium – 3 May 21
  

  
    

Conv2d: Finally Understand What Happens in the Forward Pass

  Visual and mathematical explanation of the 2D convolution layer and its arguments

  
    Reading time: 9 min read
  

  

  
    
    
  

  


So the kernel is first applied via matmul with the set stride, dilation, etc to each channel. Then those output channels are summed across channels. That is repeated for each kernel.
In Pytorch, the weights are of size:
out_channels, in_channels, kernel_dim0, kernel_dim1