<p>Without the specific context, I cannot give a definitive answer, but it's very likely that a &quot;differentiable architecture&quot; refers to a neural network that represents/computes a <a href="https://en.wikipedia.org/wiki/Differentiable_function" rel="nofollow noreferrer">differentiable function</a> (so you need to use differentiable activation functions, such as the <em>sigmoid</em>), i.e. you can take the partial derivatives of the <a href="https://en.wikipedia.org/wiki/Loss_function" rel="nofollow noreferrer">loss function</a> with respect to each parameter/weight of the neural network, so you can use <a href="http://neuralnetworksanddeeplearning.com/chap2.html" rel="nofollow noreferrer">backpropagation</a> to find the gradient of the loss function, consequently, you can train this neural network with <a href="https://www.youtube.com/watch?v=IHZwWFHWa-w" rel="nofollow noreferrer">gradient descent</a>, which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.</p>
<p>Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.</p>

CC BY-SA 4.0
2021-07-15T01:17:36.737Z
2021-07-15T01:17:36.737Z
nbro
<p>I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term &quot;Differentiable Architecture&quot;, for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is</p>
<p>What does &quot;differentiable architecture&quot; mean?</p>

CC BY-SA 4.0
2021-07-14T18:02:21.707Z
Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context?
nbro
<neural-networks><deep-learning><convolutional-neural-networks><terminology><backpropagation>
What does "differentiable architecture" mean?
Isaac Sady
# What does "differentiable architecture" mean?

(Asked by: username_0 on 2021-07-14)

I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is

What does "differentiable architecture" mean?

**Tags:** neural-networks, deep-learning, convolutional-neural-networks, terminology, backpropagation

> **Comments:**
> **username_1**: Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context?  

---

## 1 answers

### Answer by username_1 on 2021-07-15. Score: 1

Without the specific context, I cannot give a definitive answer, but it's very likely that a "differentiable architecture" refers to a neural network that represents/computes a [differentiable function](https://en.wikipedia.org/wiki/Differentiable_function) (so you need to use differentiable activation functions, such as the *sigmoid*), i.e. you can take the partial derivatives of the [loss function](https://en.wikipedia.org/wiki/Loss_function) with respect to each parameter/weight of the neural network, so you can use [backpropagation](http://neuralnetworksanddeeplearning.com/chap2.html) to find the gradient of the loss function, consequently, you can train this neural network with [gradient descent](https://www.youtube.com/watch?v=IHZwWFHWa-w), which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.

Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.


---

