Answers.Body: <p>Probably the simplest way to search for an image with the highest probability of being a cat is to use a technique similar to Deep Dream:</p>
  
  <ul>
  <li><p>Load the network for training, but freeze all the network weights</p></li>
  <li><p>Create a random input image, and connect it to the network as a "variable" i.e. data that can be changed through training</p></li>
  <li><p>Set a loss function based on maximising the pre-sigmoid value in the last layer (this is easier to handle than working with 0.999 etc probability)</p></li>
  <li><p>Train using backpropagation, but instead of using gradients to change the weights, back propagate all the way to the input layer and use gradients to change the input image.</p></li>
  <li><p>Typically you will also want to normalise the input image between iterations.</p></li>
  </ul>
  
  <p>There is a good chance that the ideal input you find which triggers "maximum catness" will be a very noisy jumbled mess of cat-related features. You may be able to encourage something more visually apppealing, or at least less noisy, by adding a little movement - e.g. minor blurring, or a slight zoom (then crop) between each iteration. At that point, it becomes a more an artistic endeavour than mathematical.</p>
  
  <p>Here is something I produced using some TensorFlow Deep Dream code plus zooming and blurring to encourage larger scale features to dominate:</p>
  
  <p><a href="https://i.sstatic.net/nPNHG.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/nPNHG.jpg" alt="enter image description here"></a></p>
  
  <p>Technically the above maximises a single internal feature map of a CNN, not a class probability, but it is the same thing conceptually.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-19T20:42:31.120Z
Answers.LastActivityDate: 2019-09-19T20:42:31.120Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <p>In deep networks there is actually a wide variety of solutions to the problem, but if you need to find one, any easy way to do this is just through normal optimization schemes<br>
  <span class="math-container">$$\hat x = argmin_x \ L(y,x)$$</span><br>
  where <span class="math-container">$L(y,x)$</span> is your loss function. Since ANN's are generally differentiable you can optimize this iteratively with some form gradient descent scheme:<br>
  <span class="math-container">$$x^{i+1} \leftarrow x^{i} - \lambda \nabla_{x^i}L(y,x^i)$$</span><br>
  where <span class="math-container">$\lambda$</span> is your learning rate.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-19T20:25:40.720Z
Answers.LastActivityDate: 2019-09-19T20:25:40.720Z
Answers.UserDisplayName: mshlis
Body: <p>Let's assume we have an ANN which takes a vector <span class="math-container">$x\in R^D$</span>, representing an image, and classifies it over two classes. The output is a vector of probabilities <span class="math-container">$N(x)=(p(x\in C_1), p(x\in C_2))^T$</span> and we pick <span class="math-container">$C_1$</span> iff <span class="math-container">$p(x\in C_1) \geq 0.5$</span>. Let the two classes be <span class="math-container">$C_1= \texttt{cat}$</span> and <span class="math-container">$C_2= \texttt{dog}$</span>. Now imagine we want to extract this ANN's idea of ideal cat by finding <span class="math-container">$x^* = argmax_x N(x)_1$</span>. How would we proceed? I was thinking about solving <span class="math-container">$\nabla_xN(x)_1=0$</span>, but I don't know if this makes sense or if it is solvable.</p>
  
  <p><strong>In short, how do I compute the input which maximizes a class-probability?</strong></p>
  
Tags: <neural-networks><generative-model>
Title: How can we find find the input image which maximizes the class-probability for an ANN?
UserDisplayName: olinarr
ThreadText: # How can we find find the input image which maximizes the class-probability for an ANN?
  
  (Asked by: username_0 on 2019-09-19)
  
  Let's assume we have an ANN which takes a vector $x\in R^D$, representing an image, and classifies it over two classes. The output is a vector of probabilities $N(x)=(p(x\in C\_1), p(x\in C\_2))^T$ and we pick $C\_1$ iff $p(x\in C\_1) \geq 0.5$. Let the two classes be $C\_1= \texttt{cat}$ and $C\_2= \texttt{dog}$. Now imagine we want to extract this ANN's idea of ideal cat by finding $x^\* = argmax\_x N(x)\_1$. How would we proceed? I was thinking about solving $\nabla\_xN(x)\_1=0$, but I don't know if this makes sense or if it is solvable.
  
  **In short, how do I compute the input which maximizes a class-probability?**
  
  **Tags:** neural-networks, generative-model
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-09-19. Score: 4 | Accepted answer
  
  In deep networks there is actually a wide variety of solutions to the problem, but if you need to find one, any easy way to do this is just through normal optimization schemes  
  $$\hat x = argmin\_x \ L(y,x)$$  
  where $L(y,x)$ is your loss function. Since ANN's are generally differentiable you can optimize this iteratively with some form gradient descent scheme:  
  $$x^{i+1} \leftarrow x^{i} - \lambda \nabla\_{x^i}L(y,x^i)$$  
  where $\lambda$ is your learning rate.
  
  
  ---
  
  ### Answer by username_1 on 2019-09-19. Score: 4
  
  Probably the simplest way to search for an image with the highest probability of being a cat is to use a technique similar to Deep Dream:
  
  * Load the network for training, but freeze all the network weights
  * Create a random input image, and connect it to the network as a "variable" i.e. data that can be changed through training
  * Set a loss function based on maximising the pre-sigmoid value in the last layer (this is easier to handle than working with 0.999 etc probability)
  * Train using backpropagation, but instead of using gradients to change the weights, back propagate all the way to the input layer and use gradients to change the input image.
  * Typically you will also want to normalise the input image between iterations.
  
  There is a good chance that the ideal input you find which triggers "maximum catness" will be a very noisy jumbled mess of cat-related features. You may be able to encourage something more visually apppealing, or at least less noisy, by adding a little movement - e.g. minor blurring, or a slight zoom (then crop) between each iteration. At that point, it becomes a more an artistic endeavour than mathematical.
  
  Here is something I produced using some TensorFlow Deep Dream code plus zooming and blurring to encourage larger scale features to dominate:
  
  [![enter image description here](https://i.sstatic.net/nPNHG.jpg)](https://i.sstatic.net/nPNHG.jpg)
  
  Technically the above maximises a single internal feature map of a CNN, not a class probability, but it is the same thing conceptually.
  
  
  ---
  
  