I must admit that I don’t even know what that bit of code might try to achieve (I prefer .requires_grad_(False), but that obviously is in conflict with takimg the grad?). Maybe just leaving the nets as they are is a good strategy or you can describe what you are trying to compute or how it compares to your initial posting.
Best regards
Thomas