Thanks ptrblck! This does remove the RuntimeError. But the two functions still aren’t leading to the same result, which they should do. I’ve verified in a few ways that introducing the 2-parameter nn.Module instead of an equivalent 2-parameter tensor causes the difference. Only the result for the tensor version is correct. I can also see that the solution you offered does at least change something compared to not doing the inner optimization at all. Maybe no_grad() removes the ability to differentiate through the opponent’s learning? Any other solutions?