Solved by tom in post #2 
              
              
                If you use lossa.clamp(min=threshold) it will only get a gradient when it’s above the threshold. That’s cool, but in reality you may have the other losses gradient pointing towards a direction of increasing lossa. How much this matters depends on your specific situation. 
Best regards 
Thomas