Hi Jan-Hendrik!



 jhmenke:

Unfortunately coding purely in torch is not possible, also computing the gradients seems to be impossible.


It is not unreasonable to compute your gradients numerically
(although suboptimal if you can compute them analytically).
So, in your forward pass you might compute:
    loss_np = numpy_stuff (x, y)
    <loop over x_i>:
        xdelt_i = x_i + delta
        grad_i = (numpy_stuff (xdelt, y) - loss_np) / delta
    ...

Then store (or accumulate) your grad_i to use in a custom
backward() function.
This is just a superficial sketch, and numerical differentiation is
more delicate than indicated above, but the main point is that
numerical differentiation can be a legitimate way to implement
backward().


Actually, an optimization problem is solved in the numpy function.


Note, that some optimization problems can be solved more
quickly if you give them a good initial guess.  So you might be
able to speed things up by reusing your initial solution when
re-solving your optimization multiple times for the numerical
differentiation.
(And, as Alban noted, some optimization algorithms naturally give
you – or can be relatively cheaply modified to give you – the
gradients you will need for your custom backward().)
But yes, just to reiterate what Alban said:  You will have to
implement a custom backward() one way or another (or
implement your custom loss function entirely with differentiable
pytorch tensor operations).
Best.
K. Frank