If you consider your network forward goes from top to bottom, the ce_loss is below the function that reverses the gradients. So it is expected not to be influenced by it.
You will see it’s effect on the gradients of the elements above it (like the output of cls_fn or input).