title: Use tf.layers.batch_normalization to preprocess inputs for SELU activation function?
body: <p>The SELU activation function (<a href="https://github.com/bioinf-jku/SNNs/blob/master/selu.py" rel="nofollow noreferrer">https://github.com/bioinf-jku/SNNs/blob/master/selu.py</a>) requires the input to be normalized to have the mean value of 0.0 and the variance of 1.0. Therefore, I tried to apply <code>tf.layers.batch_normalization</code> (<code>axis=-1</code>) on the raw data to meet that requirement. The raw data in each batch have the shape of <code>[batch_size, 15]</code>, where 15 refers to the number of features. The graph below shows the variances of 5 of these features returned from <code>tf.layers.batch_normalization</code> (~20 epochs). They are not all close to 1.0 as expected. The mean values are not all close to 0.0 as well (graphs not shown). </p>
  
  <p>How should I get the 15 features all normalized independently (I expect every feature after normalization will have mean = 0 and var = 1.0)? </p>
  
  <p><a href="https://i.stack.imgur.com/8EcQg.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8EcQg.png" alt="enter image description here"></a></p>