Thanks. But LayerNorm computes the mean/var over the neruons and they are computed for both training and test. I still want to compute the mean/var for each neuron over the batches only in training and fixed for test.