Hey there,
i want to implement something probably related to :

  
    
    
    Is it possible to use gradient w.r.t input in loss? 
  
  
    Hi, I need to construct a loss function that uses the gradient of the output of the NN w.r.t the input, i.e. something like: 

1 output = NN(input, weight) 
2 grad_input = output.backward(retain_variables=True) 
3 loss = criterion(grad_input.grad.data, target_of_grad_input) 
4 loss.backward() 

The problem is that after step 2, I can only have the numerical value of the gradient of output w.r.t to input (i.e. grad_input). Then it is no longer a function of the weight, and as a result the loss is…
  


but i am not quite sure how to do it.
I want to have a loss with multiple parts, one of which uses the derivative of the net with respect to an input value. The background is that i want to impose characteristics for the linearisations of the net at certain values of x.
Small examples with notation for clarification:

nn(x) : R -> R  is our network
x is the input
y the true value
x_1, x_2 are some values in R
d_nn(x*) is the derivative of nn(x) w.r.t x at x*

and i want a loss which could for example look like :
a = nn(x_2) - nn(x_1)  + d_nn(x_1)*x - d_nn(x_2)*x
b = d_nn(x_1) - d_nn(x2)
loss = (a/b - y )**2
I am still new to pytorch and all help is highly appreciated!