Yeh I see.
All the ops have to be pytorch’s in order the autograd to work.
In case it is useful there is a bayesian optimization library for pytorch BoTorch · Bayesian Optimization in PyTorch 1
I’m afraid I don’t use theano. But in short the story is that pytorch’s autograd generates and backpropagates gradients automatically. To do so you have to use pytorch ops (as they already have backprop and gradients coded.) If you want to use 3rd party libraries which aren’t based in pytorch you should implement their backward function the autograd to work.
You can implement a custom backward function following this tutorial.
https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html
Altough I don’t really know if jit supports that.