Hi James,
Thank you for your comments!  I actually ended up not going with torch.fx, but modified the deep net in the original source code I used. But it is great to know torch.fx does not preserve the context managers.