TonyTr:

CE will output a loss of 0 when both of inputs are the same


The PyTorch implementation of Cross Entropy loss does not accept two identical inputs: it expects the shapes of the input and target to be different. Is this not the CE that you are talking about?