Hi everyone,
I have a model trained in Pytorch, which has been serialized and imported in C++ for inference.
Let’s say the NN has n_in inputs and n_out outputs. For every sample, I would like to access the derivative of all the outputs with respect to the inputs. What I am doing is the following:
  //define input
  torch::Tensor x = torch::rand( {1,n_in} );
  x.set_requires_grad(true);

  //convert to Ivalue
  std::vector<torch::jit::IValue> inputs;
  inputs.push_back( x );

  //calculate output
  torch::Tensor output = model.forward( inputs ).toTensor();

  //calculate derivatives of every output j with respect to x
  for(unsigned j=0; j<n_out; j++){
    output[0][j].backward();         // 0 because is a batch of size 1
    torch::Tensor der_j = x.grad()
    /*
    ...
    */
  }

The problem is that in this way the gradients are accumulated into x.grad at each call of output[0][j].backward(). Since I am not using an optimizer I cannot call zero_grad() on it. Should I access the gradient of x after each backward call and manually set to zero? Or are there better ways of doing this?
Thank you,
Luigi