Supporting the nn.Module approach would mean going down the same issue as the checkpointing module we have. Which has severe limitations forcing the user to user backward() and can lead to unexpected behavior wrt things requiring gradients.