Answers.Body: <p>Without the specific context, I cannot give a definitive answer, but it's very likely that a &quot;differentiable architecture&quot; refers to a neural network that represents/computes a <a href="https://en.wikipedia.org/wiki/Differentiable_function" rel="nofollow noreferrer">differentiable function</a> (so you need to use differentiable activation functions, such as the <em>sigmoid</em>), i.e. you can take the partial derivatives of the <a href="https://en.wikipedia.org/wiki/Loss_function" rel="nofollow noreferrer">loss function</a> with respect to each parameter/weight of the neural network, so you can use <a href="http://neuralnetworksanddeeplearning.com/chap2.html" rel="nofollow noreferrer">backpropagation</a> to find the gradient of the loss function, consequently, you can train this neural network with <a href="https://www.youtube.com/watch?v=IHZwWFHWa-w" rel="nofollow noreferrer">gradient descent</a>, which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.</p>
  <p>Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-15T01:17:36.737Z
Answers.LastActivityDate: 2021-07-15T01:17:36.737Z
Answers.UserDisplayName: nbro
Body: <p>I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term &quot;Differentiable Architecture&quot;, for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is</p>
  <p>What does &quot;differentiable architecture&quot; mean?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-14T18:02:21.707Z
Comments.Text: Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context?
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><convolutional-neural-networks><terminology><backpropagation>
Title: What does "differentiable architecture" mean?
UserDisplayName: Isaac Sady
ThreadText: # What does "differentiable architecture" mean?
  
  (Asked by: username_0 on 2021-07-14)
  
  I'm currently reading a paper that uses CNN's as a base approach to solving some image classification issues and I've found that they kept mentioning the term "Differentiable Architecture", for which I have no idea about its meaning, as I'm new to this world of Deep Learning, Neural Networks, etc., so to sum up my question is
  
  What does "differentiable architecture" mean?
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, terminology, backpropagation
  
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE. Could you please share the link to or name of the paper you were reading for more context?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-07-15. Score: 1
  
  Without the specific context, I cannot give a definitive answer, but it's very likely that a "differentiable architecture" refers to a neural network that represents/computes a [differentiable function](https://en.wikipedia.org/wiki/Differentiable_function) (so you need to use differentiable activation functions, such as the *sigmoid*), i.e. you can take the partial derivatives of the [loss function](https://en.wikipedia.org/wiki/Loss_function) with respect to each parameter/weight of the neural network, so you can use [backpropagation](http://neuralnetworksanddeeplearning.com/chap2.html) to find the gradient of the loss function, consequently, you can train this neural network with [gradient descent](https://www.youtube.com/watch?v=IHZwWFHWa-w), which is a numerical/iterative optimization algorithm for finding a (local) minimum of a function.
  
  Most architectures you will find around are differentiable. In fact, gradient descent is the most widely used algorithm for training neural networks nowadays.
  
  
  ---
  
  