Answers.Body: <p>(Good) GANs are highly specific. If you had a specific domain in mind, e.g. just human faces then you could use something like a <a href="https://arxiv.org/abs/1711.11586" rel="nofollow noreferrer">Bicycle GAN</a>. On top of a Generator (&amp; Discriminator) this includes an Encoder, which would take your image, encode it into a latent variable <code>z</code>, which you could then feed into the Generator, perhaps with extra noise to get variance. To my knowledge there don't exist any good pretrained variants so you would have to train it yourself.</p>
  <p>On the other hand, I think that Diffusion models are better suited for this particular task, they are trained to denoise a noisy image. So all you would have to do is take your original, add (some) noise and then denoise it to get a similar image.</p>
  <p>Both of these would produce an image similar not only in content but also in style, if you actually cared about variance in style and keeping only the content, I would perhaps suggest:</p>
  <ul>
  <li>feeding the original image into a <a href="https://huggingface.co/tasks/image-to-text" rel="nofollow noreferrer">Captioning network</a></li>
  <li>feeding the caption into a text-to-image Diffusion model</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-31T09:36:35.293Z
Answers.LastActivityDate: 2023-08-31T09:36:35.293Z
Answers.UserDisplayName: Ronald
Answers.Body: <p>The closest literature to what you're suggesting is indeed related to GANs, specifically to arithmetic performed on the latent space learned by generators.</p>
  <p>Check <a href="https://arxiv.org/pdf/1511.06434.pdf" rel="nofollow noreferrer">Unsupervised Representation Learning with deep convolutional generative adversarial networks.</a></p>
  <p>The idea suggested in the paper is to &quot;tune&quot; specific characteristic of the generated images by discarding specific filters learned by the generator. For example, by removing filters related to specific objects like windows, we could generate images of rooms without windows (see second image below, top row: images generated with all filters, bottom row: images generated without window filters).</p>
  <p><a href="https://i.sstatic.net/QNeCL.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QNeCL.png" alt="enter image description here" /></a>
  <a href="https://i.sstatic.net/h2T6L.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/h2T6L.png" alt="top row: images generated with all filters, bottom row: images generated without window filters" /></a></p>
  <p>Other <a href="https://openaccess.thecvf.com/content_CVPR_2020/papers/Shen_Interpreting_the_Latent_Space_of_GANs_for_Semantic_Face_Editing_CVPR_2020_paper.pdf" rel="nofollow noreferrer">papers</a> applied and refined the same idea, showing that you can tune characteristic of faces like age/glasses and so on.</p>
  <p><a href="https://i.sstatic.net/Kpf0H.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Kpf0H.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-03T16:13:47.157Z
Answers.LastActivityDate: 2022-03-03T16:13:47.157Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>Are there some known neural networks that, given an input image, can generate a <strong>similar image</strong>, with the same topic?</p>
  <p>Example: input = a photo of a cat on a green table, output = a generated photo of another cat on another green table.</p>
  <p>Example 2: input = a portrait of a man with glasses and a beard, output = a portrait of a generated person with similar glasses / beard (see &quot;ThisPersonDoesNotExist&quot;).</p>
  <p>I imagine it is possible with a <a href="https://en.wikipedia.org/wiki/Generative_adversarial_network" rel="nofollow noreferrer">GAN</a>, but more precisely which kind of architecture?</p>
  
Tags: <neural-networks><convolutional-neural-networks><generative-adversarial-networks><image-generation><model-request>
Title: Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?
UserDisplayName: logijaz
ThreadText: # Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?
  
  (Asked by: username_0 on 2022-03-03)
  
  Are there some known neural networks that, given an input image, can generate a **similar image**, with the same topic?
  
  Example: input = a photo of a cat on a green table, output = a generated photo of another cat on another green table.
  
  Example 2: input = a portrait of a man with glasses and a beard, output = a portrait of a generated person with similar glasses / beard (see "ThisPersonDoesNotExist").
  
  I imagine it is possible with a [GAN](https://en.wikipedia.org/wiki/Generative_adversarial_network), but more precisely which kind of architecture?
  
  **Tags:** neural-networks, convolutional-neural-networks, generative-adversarial-networks, image-generation, model-request
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-08-31. Score: 2
  
  (Good) GANs are highly specific. If you had a specific domain in mind, e.g. just human faces then you could use something like a [Bicycle GAN](https://arxiv.org/abs/1711.11586). On top of a Generator (& Discriminator) this includes an Encoder, which would take your image, encode it into a latent variable `z`, which you could then feed into the Generator, perhaps with extra noise to get variance. To my knowledge there don't exist any good pretrained variants so you would have to train it yourself.
  
  On the other hand, I think that Diffusion models are better suited for this particular task, they are trained to denoise a noisy image. So all you would have to do is take your original, add (some) noise and then denoise it to get a similar image.
  
  Both of these would produce an image similar not only in content but also in style, if you actually cared about variance in style and keeping only the content, I would perhaps suggest:
  
  * feeding the original image into a [Captioning network](https://huggingface.co/tasks/image-to-text)
  * feeding the caption into a text-to-image Diffusion model
  
  
  ---
  
  ### Answer by username_2 on 2022-03-03. Score: 1
  
  The closest literature to what you're suggesting is indeed related to GANs, specifically to arithmetic performed on the latent space learned by generators.
  
  Check [Unsupervised Representation Learning with deep convolutional generative adversarial networks.](https://arxiv.org/pdf/1511.06434.pdf)
  
  The idea suggested in the paper is to "tune" specific characteristic of the generated images by discarding specific filters learned by the generator. For example, by removing filters related to specific objects like windows, we could generate images of rooms without windows (see second image below, top row: images generated with all filters, bottom row: images generated without window filters).
  
  [![enter image description here](https://i.sstatic.net/QNeCL.png)](https://i.sstatic.net/QNeCL.png)
  [![top row: images generated with all filters, bottom row: images generated without window filters](https://i.sstatic.net/h2T6L.png)](https://i.sstatic.net/h2T6L.png)
  
  Other [papers](https://openaccess.thecvf.com/content_CVPR_2020/papers/Shen_Interpreting_the_Latent_Space_of_GANs_for_Semantic_Face_Editing_CVPR_2020_paper.pdf) applied and refined the same idea, showing that you can tune characteristic of faces like age/glasses and so on.
  
  [![enter image description here](https://i.sstatic.net/Kpf0H.png)](https://i.sstatic.net/Kpf0H.png)
  
  
  ---
  
  