Thank you again for the explanation @tom @googlebot !
Something like this would solve my problem just fine:
c = MyModule()
# Tell tracer to also track some parameters
forward = jit.trace(c, (inputs,), track=c.parameters())

To me tracing looks like the more elegant solution if the requirements for tracing are met, which they are for me except that I need to record parameters requires_grad.
I guess currently there is nothing much I can do except either change my code to support scripting or not use the jit.