If i understood right, you are doing a dummy training with a constant image and heat map ? I see at least one issue, the heat map is not normalized. When defining your dummy heat map do this:
heatmap = torch.rand(1,16,1,160,160).to(DEVICE)
heatmap = heatmap/torch.sum(dim=(2,3,4), keepdim=True)

But there is maybe something else that disturb the training… Still looking