Solved by seungjun in post #4 
              
              
                Ah, sorry for the misunderstanding. 
You can try below. 
C = (A.unfold(2,80,1).unfold(3,80,1) - B.expand(b,c,21,21,80,80)).mean(dim=(-2,-1))

# variable version
_, _, H, W = A.shape
_, _, h, w = B.shape
C = (A.unfold(2,h,1).unfold(3,w,1) - B.expand(b,c,H-h+1,W-w+1,h,w)).mean(dim=(-2,-1))

The keys a…