my loss is a KL(A(x)|| B(A(x))), B is well trained and fixed, and only  A is needed training… (its called educational networks I think) could it because of KL is too complex to use autograd 
thanks
My understanding is that to do derivative KL w.r.t. theta_A  (write KL a function of A(x) and B(A(x)) to make it simpler)
even B is not allow derivative, we should still get derivative from two parts
something like ( sorry for the messy function) :
( d KL(A(x),B(A(x))/ dA ) * ( d A / d theta_A) + (d KL(A(x),B(A(x)) / d B)   * (d B/ dA)  *  (dA / d theta_A)
currently, it seemed I have to give up the second term to make it work