If I understand you correctly, would it be the way to subtract B from part of A?
You can specify corresponding index ranges if the dimension lengths equal to each other.
C = A[:,:,80:100, 80:100] - B
# C shape: b x c x 20 x 20

You can further apply torch.mean afterwards by your needs.