Sounds like you want to register a parametrization -
https://pytorch.org/docs/master/generated/torch.nn.utils.parametrize.register_parametrization.html?highlight=parametrization#torch.nn.utils.parametrize.register_parametrization 4
This requires PyTorch version >= 1.9 though.