Solved by KFrank in post #6 
              
              
                Hi Jan-Hendrik! 

It is not unreasonable to compute your gradients numerically 
(although suboptimal if you can compute them analytically). 
So, in your forward pass you might compute: 
    loss_np = numpy_stuff (x, y)
    <loop over x_i>:
        xdelt_i = x_i + delta
        grad_i = (numpy_stuff …