You could move the condition out of the weight_init method and apply it to a loop iteratint all named children manually. If the condition is met, you could them call into the weight_init, which could check for the type of the method or could even apply the torch.nn.init method directly on the child module.