Ah, sorry for the misunderstanding.
You can try below.
C = (A.unfold(2,80,1).unfold(3,80,1) - B.expand(b,c,21,21,80,80)).mean(dim=(-2,-1))

# variable version
_, _, H, W = A.shape
_, _, h, w = B.shape
C = (A.unfold(2,h,1).unfold(3,w,1) - B.expand(b,c,H-h+1,W-w+1,h,w)).mean(dim=(-2,-1))

The keys are torch.Tensor.unfold 1 and torch.Tensor.expand 2.
The unfold function extracts sliding window and expand function views the tensor in expanded size without copying values.