title: TensorFlow: Compute Hessian matrix (and higher order derivatives)
body: <p>I would like to be able to compute higher order derivatives for my loss function. At the very least I would like to be able to compute the Hessian matrix. At the moment I am computing a numerical approximation to the Hessian but this is more expensive, and more importantly, as far as I understand, inaccurate if the matrix is ill-conditioned (with very large condition number).</p>
  
  <p>Theano implements this through symbolic looping, see <a href="http://deeplearning.net/software/theano/library/scan.html" rel="noreferrer">here</a>, but Tensorflow does not seem to support symbolic control flow yet, see <a href="https://github.com/samjabrahams/tensorflow-white-paper-notes/commit/156736904eb48a0a44de4c23b227ae938e37e654" rel="noreferrer">here</a>. A similar issue has been raised on TF github page, see <a href="https://github.com/tensorflow/tensorflow/issues/675" rel="noreferrer">here</a>, but it looks like nobody has followed up on the issue for a while.</p>
  
  <p>Is anyone aware of more recent developments or ways to compute higher order derivatives (symbolically) in TensorFlow?</p>