What the backward computes really is a vector Jacobian product, rather than the full Jacobian, for efficiency reasons, and also for the fact that most people really only care about the gradients of parameters w.r.t. a single scalar metric.
The current conv2d CPU backward is here https://github.com/pytorch/pytorch/blob/22e3b2c9c369c5fb44476eb538fa0a308df94eff/aten/src/THNN/generic/SpatialConvolutionMM.c#L247-L414 117