OK, I performed network surgery, redefined the network and only loaded the state_dict from the modules that existed (removing the method that had a variable function). Still no dice. I added requires_grad=True to the tensors, but still there are parameters in the network without a gradient. Not sure what is going on, but it feels like there might be something wrong. (note that everything is OK when I am training from scratch, I can load models back in, etc).