hmmm so I’m struggling to visualize why numpy is even called? If your loss is calculate purely by a nn.Module why do you need to detach it to numpy?



 colinrsmall:

The actual code is really, really messy since the discriminator model is implemented in Tensorflow (don’t ask).


I assume this is the reason as to why you’re detaching to numpy? So, you can pass your values to TF? I don’t think that’ll work as how is PyTorch meant to track TF’s operations? Also, TF and PyTorch are structured completely differently. PyTorch is dynamic whereas TF is static. Is it not possible to re-write the TF code in PyTorch?
Also, torch.autograd.Function is a potential avenue for adding in additional functional to PyTorch with the python APi but there are some caveats to it, for example, you won’t be able to JIT your network. And, you’ll need to manually define your derivatives which gets extremely messy if your operation involves Linear algebra operations.