Ok I figured this out. For some reason the weights need to be detached and reassigned as a Parameter with grad re-enabled.
new_input_conv.weight = nn.Parameter(new_input_conv.weight.detach().requires_grad_(True))