Is Glorot/He-style variance-preserving *regularization* a known thing? In the context of deep learning, layer weight initialization techniques of the Glorot/He family strive to select initial random weight values in such a way that the variance of each layer's output is equal to the variance of its input. This helps avoid vanishing/exploding gradient issues and improves training stability.
It seems likely to me that this is a desirable property to maintain not only initially, but also during training, as a regularization constraint via an explicit loss term nudging weight distribution to the optimal distribution mandated by Glorot/He.
Has this been studied?


Has this been studied?

Yes, it has been studied.  You are describing one of the goals of batchnorm.  Other methods exist as well - see @Sycorax's answer for another.
So in fact, it has been implemented in every major library, and used very extensively in research models and in deployed models.

variance of each layer's output is equal to the variance of its input

Maintaining activation variances over the course of gradient descent is one of the two mechanisms behind Batch Normalization (a.k.a. batchnorm.)  The other mechanism is to maintain activation means.  With batchnorm, each batch of training data, a translation and scale parameter is learned for each layer, such that the activations at each layer are re-centered and re-scaleed to track zero mean and unit variance.

as a regularization constraint.

Indeed, batchnorm has a regularizing effect.  The original goal of batchnorm was to reduce a phenomenon dubbed, "internal covariate shift," but more recent research suggests that it has a regularizing effect by smoothing the gradient steps, hence smoothing the effective loss function.
Figure 1 (c) of the latter paper shows layer 3 and layer 11 VGG activation distributions over the course of training, with and without batchnorm:

With batchnorm (blue), wee see that the distributions stay more consistent over training, especially in the tail behaviour, and especially in very early training.  In the bottom left, note how quickly the activation distributions narrow into tight variance and stay that way for the duration of training.
