Answers.Body: <p>It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function <em>is</em> differentiable*, and usually trivially so.</p>
  
  <p>For example:</p>
  
  <ul>
  <li><p>If an average pooling layer has inputs <span class="math-container">$z$</span> and outputs <span class="math-container">$a$</span>, and each output is average of 4 inputs then <span class="math-container">$\frac{da}{dz} = 0.25$</span> (if pooling layers overlap it gets a little more complicated, but you just add things up where they overlap). </p></li>
  <li><p>A max pooling layer has <span class="math-container">$\frac{da}{dz} = 1$</span> for the maximum z, and <span class="math-container">$\frac{da}{dz} = 0$</span> for all others.</p></li>
  </ul>
  
  <p>A pooling layer usually has no learnable parameters, but if you know the gradient of a function at its outputs, you can assign gradient correctly to its inputs using the chain rule. That is essentially all that back propagation is, the chain rule applied to the functions of a neural network.</p>
  
  <p>To answer your question more directly:</p>
  
  <blockquote>
    <p>Can non-differentiable layer be used in a neural network, if it's not learned?</p>
  </blockquote>
  
  <p>No. </p>
  
  <p>There is one exception: If this layer appears directly after the input, then as it has no parameters to learn, and you generally do not care about the gradient of the input data, so you can have a non-differentiable function there. However, this is just the same as transforming your input data in some non-differentiable way, and training the NN with that transformed data instead.</p>
  
  <hr>
  
  <p>* Technically there are some discontinuities in the gradient of a max function (where any two inputs are equal). However, this is not a problem in practice, as the gradients are well behaved close to these values. When you can safely do this or not is probably the topic of another question.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-05T01:19:54.103Z
Answers.Text: https://math.stackexchange.com/a/2885265
Answers.UserDisplayName: user76284
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-08T17:00:29.943Z
Answers.Text: To describe a function as differentiable means that it is differentiable at all points in its domain, and this is not true of the max-pooling function. If $f(x) = \max_i f_i(x)$, then $f$ is typically *not* a differentiable function, even if each function $f_i$ is. If we look at the graph of $f$ we will see some sharp corners.
Answers.UserDisplayName: littleO
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-08T20:08:19.910Z
Answers.Text: @littleO: Those sharp points are not important to gradient descent by backpropagation in practice, and you can take whichever of the two overlapping values you prefer as the value of the "gradient" - everything will work fine. I will try to add that detail somehow without changing the flow of the answer.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-08T20:09:41.830Z
Answers.Text: This answer is related to the math.stackexchange question here:  https://math.stackexchange.com/questions/2837737/neural-networks-and-the-chain-rule
Answers.UserDisplayName: NicNic8
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-08T20:19:36.233Z
Answers.Text: @NicNic8: Yes, but note that the question here is not really about the maths - the OP thought that the function was not differentiable at all, whilst it is entirely possible to use the chain rule in domains of the input functions that are differentiable. Further to that, it is not even very important in this case if we hit a non-differentiable point, we can safely patch it. That may not be the case for all functions or for all situations where you want to use the chain rule . . . but that's a different question
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-08T21:53:36.730Z
Answers.Text: Thanks. I think it's worth being careful about this point, because there are examples where gradient descent (with exact like search) fails to converge to a local minimizer despite never encountering a nondifferentiable point along the way. (Such an example can be found in Vandenberghe's 236c notes.) So, it seems to me that there is a serious theory question about how well we can expect gradient descent to perform when the objective is nondifferentiable, even if the objective is differentiable almost everywhere. If the goal is to grock deep learning, I think this point should be grappled with.
Answers.UserDisplayName: littleO
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-09T01:50:18.427Z
Answers.Text: @NeilSlater I very much appreciate your efforts here.  I also think it's important to be careful when saying things like "it is not even very important in this case if we hit a non-differentiable point".  Many theorems regarding the convergence and the rate of convergence of gradient descent aren't valid unless the objective function is differentiable.
Answers.UserDisplayName: NicNic8
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-24T18:58:15.103Z
Answers.LastActivityDate: 2019-01-08T20:12:18.537Z
Answers.LastEditDate: 2019-01-08T20:12:18.537Z
Answers.UserDisplayName: Neil Slater
Body: <p>For example, AFAIK, the pooling layer in a CNN is not differentiable, but it can be used because it's not learning. Is it always true?</p>
  
Tags: <neural-networks><machine-learning><backpropagation><gradient-descent><pooling>
Title: Can non-differentiable layer be used in a neural network, if it's not learned?
UserDisplayName: RedRus
ThreadText: # Can non-differentiable layer be used in a neural network, if it's not learned?
  
  (Asked by: username_0 on 2018-08-24)
  
  For example, AFAIK, the pooling layer in a CNN is not differentiable, but it can be used because it's not learning. Is it always true?
  
  **Tags:** neural-networks, machine-learning, backpropagation, gradient-descent, pooling
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-08-24. Score: 9 | Accepted answer
  
  It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function *is* differentiable\*, and usually trivially so.
  
  For example:
  
  * If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a little more complicated, but you just add things up where they overlap).
  * A max pooling layer has $\frac{da}{dz} = 1$ for the maximum z, and $\frac{da}{dz} = 0$ for all others.
  
  A pooling layer usually has no learnable parameters, but if you know the gradient of a function at its outputs, you can assign gradient correctly to its inputs using the chain rule. That is essentially all that back propagation is, the chain rule applied to the functions of a neural network.
  
  To answer your question more directly:
  
  > Can non-differentiable layer be used in a neural network, if it's not learned?
  
  No.
  
  There is one exception: If this layer appears directly after the input, then as it has no parameters to learn, and you generally do not care about the gradient of the input data, so you can have a non-differentiable function there. However, this is just the same as transforming your input data in some non-differentiable way, and training the NN with that transformed data instead.
  
  ---
  
  \* Technically there are some discontinuities in the gradient of a max function (where any two inputs are equal). However, this is not a problem in practice, as the gradients are well behaved close to these values. When you can safely do this or not is probably the topic of another question.
  
  > **Comments:**
  > **username_3**: To describe a function as differentiable means that it is differentiable at all points in its domain, and this is not true of the max-pooling function. If $f(x) = \max_i f_i(x)$, then $f$ is typically *not* a differentiable function, even if each function $f_i$ is. If we look at the graph of $f$ we will see some sharp corners.  
  > **username_1**: @username_3: Those sharp points are not important to gradient descent by backpropagation in practice, and you can take whichever of the two overlapping values you prefer as the value of the "gradient" - everything will work fine. I will try to add that detail somehow without changing the flow of the answer.  
  > **username_4**: This answer is related to the math.stackexchange question here:  https://math.stackexchange.com/questions/2837737/neural-networks-and-the-chain-rule  
  > **username_1**: @username_4: Yes, but note that the question here is not really about the maths - the OP thought that the function was not differentiable at all, whilst it is entirely possible to use the chain rule in domains of the input functions that are differentiable. Further to that, it is not even very important in this case if we hit a non-differentiable point, we can safely patch it. That may not be the case for all functions or for all situations where you want to use the chain rule . . . but that's a different question  
  > **username_3**: Thanks. I think it's worth being careful about this point, because there are examples where gradient descent (with exact like search) fails to converge to a local minimizer despite never encountering a nondifferentiable point along the way. (Such an example can be found in Vandenberghe's 236c notes.) So, it seems to me that there is a serious theory question about how well we can expect gradient descent to perform when the objective is nondifferentiable, even if the objective is differentiable almost everywhere. If the goal is to grock deep learning, I think this point should be grappled with.  
  > **username_4**: @username_1 I very much appreciate your efforts here.  I also think it's important to be careful when saying things like "it is not even very important in this case if we hit a non-differentiable point".  Many theorems regarding the convergence and the rate of convergence of gradient descent aren't valid unless the objective function is differentiable.  
  > **username_2**: https://math.stackexchange.com/a/2885265  
  
  ---
  
  