Interesting! That worked really well in the example.
I’m not quite getting the intuition for what this does/why it works. What’s your understanding? I think I understand why you keep the last axis, and what this would do with SGD.  But it’s less clear when applied through one of these more complex optimizers. Can you summarize your understanding of it (without using the word “Lipschitzness” ).