Answers.Body: <p>Well, you can specify a custom gradient by either being just the identity (i.e. returning the inputs in the gradient scope) or computing the gradient by hand if you know that expression.</p>
  <p>Otherwise, you can approximate the gradient of non-differentiable functions by setting-up a <a href="https://www.cl.uni-heidelberg.de/statnlpgroup/blog/sfge/" rel="nofollow noreferrer">score function estimator</a> also called the <a href="https://link.springer.com/content/pdf/10.1023/A:1022672621406.pdf" rel="nofollow noreferrer">REINFORCE</a> estimator: also have a read <a href="https://mpatacchiola.github.io/blog/2021/02/08/intro-variational-inference-2.html" rel="nofollow noreferrer">here</a>.</p>
  <p>The <em>finite difference method</em> that you mentioned may work too, but it's often unstable for very small <span class="math-container">$\delta$</span> which are required in order to get good estimates.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-11T09:35:49.380Z
Answers.LastActivityDate: 2023-06-11T10:29:43.983Z
Answers.LastEditDate: 2023-06-11T10:29:43.983Z
Answers.UserDisplayName: Luca Anzalone
Answers.Body: <p>You can maybe use a similar re-parameterisation trick. Where you may approximate the gradient rather than calculating it accurately.</p>
  <p>Introduce auxiliary variables which can mimic the function an still be differentiable. For example, as you suggested in a VAE when we have to pick a random vector from a distribution, we don't actually pick a random vector. But we pick a <span class="math-container">$\mu$</span> and <span class="math-container">$\sigma$</span> (i.e mean and variance) to train on. And then supple the random vector as an output of that like random vector <span class="math-container">$z$</span> = <span class="math-container">$\mu$</span>+ <span class="math-container">$\epsilon*\sigma$</span> where <span class="math-container">$\epsilon$</span> ~ (0, 1).</p>
  <p>Kindly check this reference: <a href="https://stats.stackexchange.com/a/205336/382270">Reparameterisation trick</a>. Also kindly provide an example of your vector function so that we can construct a more definite answer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-11T22:14:32.603Z
Answers.LastActivityDate: 2023-06-11T22:14:32.603Z
Answers.UserDisplayName: Formal_this
Body: <p>I worked with Feed Forward Neural Network and VAE and understood backpropagation algorithm. Now I build a VAE network, one layer of it is a very complex vector-to-vector function <span class="math-container">$f(x)$</span> (a general 'method' in the programming sense instead of a 'math' expression).</p>
  <p>Thus, there is no gradient info for this layer. I guess one cannot train the entire network with such a gap, though other layers are differentiable. Is there any nice way to train such a network?</p>
  <p>One thing in my mind is to approximate the gradient by slightly changing <span class="math-container">$x$</span> and computing
  <span class="math-container">$$
  \frac{f(x+\delta)-f(x)}{ \delta}
  $$</span></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-11T10:57:17.600Z
Comments.Text: Your idea of approximating the gradient by slightly changing x is the definition of a derivative.
Comments.UserDisplayName: David Hoelzer
Tags: <training><backpropagation><variational-autoencoder><gradient>
Title: How to apply backpropagation when one layer of the network is a call-only function (no gradient)?
UserDisplayName: whitegreen
ThreadText: # How to apply backpropagation when one layer of the network is a call-only function (no gradient)?
  
  (Asked by: username_0 on 2023-06-11)
  
  I worked with Feed Forward Neural Network and VAE and understood backpropagation algorithm. Now I build a VAE network, one layer of it is a very complex vector-to-vector function $f(x)$ (a general 'method' in the programming sense instead of a 'math' expression).
  
  Thus, there is no gradient info for this layer. I guess one cannot train the entire network with such a gap, though other layers are differentiable. Is there any nice way to train such a network?
  
  One thing in my mind is to approximate the gradient by slightly changing $x$ and computing
  $$
  \frac{f(x+\delta)-f(x)}{ \delta}
  $$
  
  **Tags:** training, backpropagation, variational-autoencoder, gradient
  > **Comments:**
  > **username_1**: Your idea of approximating the gradient by slightly changing x is the definition of a derivative.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-06-11. Score: 2
  
  Well, you can specify a custom gradient by either being just the identity (i.e. returning the inputs in the gradient scope) or computing the gradient by hand if you know that expression.
  
  Otherwise, you can approximate the gradient of non-differentiable functions by setting-up a [score function estimator](https://www.cl.uni-heidelberg.de/statnlpgroup/blog/sfge/) also called the [REINFORCE](https://link.springer.com/content/pdf/10.1023/A:1022672621406.pdf) estimator: also have a read [here](https://mpatacchiola.github.io/blog/2021/02/08/intro-variational-inference-2.html).
  
  The *finite difference method* that you mentioned may work too, but it's often unstable for very small $\delta$ which are required in order to get good estimates.
  
  
  ---
  
  ### Answer by username_3 on 2023-06-11. Score: 1
  
  You can maybe use a similar re-parameterisation trick. Where you may approximate the gradient rather than calculating it accurately.
  
  Introduce auxiliary variables which can mimic the function an still be differentiable. For example, as you suggested in a VAE when we have to pick a random vector from a distribution, we don't actually pick a random vector. But we pick a $\mu$ and $\sigma$ (i.e mean and variance) to train on. And then supple the random vector as an output of that like random vector $z$ = $\mu$+ $\epsilon\*\sigma$ where $\epsilon$ ~ (0, 1).
  
  Kindly check this reference: [Reparameterisation trick](https://stats.stackexchange.com/a/205336/382270). Also kindly provide an example of your vector function so that we can construct a more definite answer.
  
  
  ---
  
  