Hi,
If the goal is that these samples are always properly classified, why do you use them in the model at all?
Why not filter them out before the forward based on the target? This way you won’t do useless computations in the forward/backward and will be able to use the vanilla loss function.