Solved by albanD in post #2 
              
              
                Hi, 
You will need to use the scores in a differentiable manner and give a differentiable input to the second network if you want gradients to be able to flow back. 
Note that if the inputs of the second network are integers, this most likely won’t be possible as you can’t get gradients for non-cont…