x[b,h,:,:] = torch.div(x[b,h,:,:],torch.sum(x[b,h,:,:]))

is always an inplace operation whether you operate on a clone of the tensor or not.
You can replace those for loops with this.
x = x / x.sum(dim=3).sum(dim=2).unsqueeze(2).unsqueeze(3)