markdaoust:

Annother possible solution would be for keras to implements a global “default_initializer” or something like that.


But if I remember correctly something similar didn’t pass in 2019:

  

      github.com/tensorflow/tensorflow
  

  
    
  
	  
  

  
    
      Changing the default initializer globally for tf.keras.layers 7
    

    
      
        opened 
        
          
        
        Mar 2, 2019
      
      

        
          closed 
        
          
        
        Mar 9, 2019
      
        

      
        
          
          guillaumekln
        
      
    

    
        TF 2.0
        comp:ops
        stat:awaiting tensorflower
        type:feature
    
  


  
    **System information**
- TensorFlow version: 2.0
- Are you willing to contribu…te it: No

**Describe the feature and the current behavior/state.**

For V1 `tf.layers`, one could set a default initializer to the global variable scope:

```python
with tf.variable_scope(name, initializer=...):
  ...
```

In V2 `tf.keras.layers`, the default initializer is hardcoded to `initializers.glorot_uniform()`. I currently see 2 imperfect ways to change the default initializer globally:

* change the initializer to every layers that are created **but** this is not convenient for large models
* implement a manual assignation loop **but** AFAIK there is no clean way to detect if the variable initializer is the default one or an initializer that was explicitly passed by the developer (typically a constant initialization).

**Will this change the current api? How?**

I think the most user friendly approach to this is to add an endpoint to globally change the default initializer: each layer created after this statement will use this initializer by default.

**Who will benefit with this feature?**

Anyone who wants to easily apply a global initialization strategy while still ensuring that specific initializer can be set locally.

---

Any recommendation to achieve this with the current code is of course appreciated.