I guess I was worried that if the original weights were non trianable that putting W_trainable + W_non_trainable would become non trainable…but the del would actually delete the old instance/object so that the requires_grad is set to True.