Solved by harshays in post #3 
              
              
                Hi Victor, 
I’m trying to implement a regulariser described in this paper: [2105.05612] Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization. 
The shared features are a function of the input and model parameters, so the gradient of head o…