Hi,
Why are you using loss_per_channel instead of finding the total loss of all channels?  Are you getting multi loss. For example, please explain, a prediction of [10,3,5,5] with ground truth [10,1,5,5] will work?
Best