Thank you very much, iterating over the children worked!
I’ve been trying to find recommendations on how to initialize the batchnorm weights, biases, running_mean, and running_var and the layernorm weights and biases but can’t find any recommendations, do you know if there is any special way to initialize those layers?