But if that loss term is only influenced by that layers weight, the gradient corresponding to that part of the loss will only influence that layer’s weights.
Also you still haven’t shared any formula of what you’re trying to compute so it is hard to say