Hi Truss!



 truss712:




 KFrank:

Should the update you apply to your
prediction-network parameters depend in any way on the output
of the simulation layer?


…
These parameters are updated by calculating losses by comparing simulated image with true image in discriminator.
…
this simulated image from my simulation algorithm is then compared with true image in discriminator. Losses are also calculated here.
…
Yes, simulated image generated using simulation layer (only numpy functions) will be compared with real image in discriminator stage to train it and then use discriminator to calculate losses for generator.


Reading between the lines, you are, as I understand it, trying to train
the discriminator to tell simulated images apart from real ones, and to
train the generator to generate simulation parameters that lead to
simulated images that are hard to tell apart from real ones.
So, yes, in order to update the generator “weights” you do need to
backpropagate the final discriminator loss back through the simulator.
This is the only way the generator knows whether it did a good job
of generating parameters that produce simulated images that look
realistic to the discriminator.
The simulation process is an important, non-trivial part of how the
generator “weights” produce the simulated images that are used to
calculate the final loss, so to train the generator “weights” you have
to backpropagate through it properly.
Best.
K. Frank