title: How does xavier_initializer() know the activation?
body: <p>How does tf.contrib.layers.xavier_initializer() know the activation functions?</p>
  
  <p>The standard deviation to initialize well depends on the non-linearity used. Right? How then does tf.contrib.layers.xavier_initializer() know whats going on?</p>
  
  <p>Take the following case:</p>
  
  <pre><code>W = tf.get_variable("W", shape=[784, 256],
             initializer=tf.contrib.layers.xavier_initializer())
  </code></pre>
  
  <p>This W does something to X and then the result is passed to a tanh or relu or have you. Now, the initializer is in the W. How does tensorflow figure the activation out? Or do I have to intervene, knowing the activation I am going to use?</p>
  
  <p>Looked at the arguments in <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/api_docs/python/functions_and_classes/shard9/tf.contrib.layers.xavier_initializer.md" rel="nofollow" title="tf.contrib.layers.xavier_initializer">tf.contrib.layers.xavier_initializer</a> and there I can chose unifrom or normal distribution. But that doesn't solve it, right?</p>