Well, I found a solution that worked for me.
We can first torch.jit.script submodule A (maybe combine with torch.jit.ignore to make it scriptable, just to ensure that it will be skipped by torch.jit.trace), and then torch.jit.trace the whole model.
Still looking for a “solution” for the title.