Thank you, your words inspired me.
I searched weight normalization and I don’t know is that what you said.
Seems like Weight Normalization is helpful to training, and was introduced in this paper.
Is there any evidence that torch implemented this?
And looks like weight normalization is an optional operation in models.
Is there any evidence showing that a network’s weight will be normalized