Answers.Body: <p>If I understand correctly, you are looking for the image-to-image translation method. For example take a look at pix2pix GAN (<a href="https://arxiv.org/abs/1611.07004" rel="nofollow noreferrer">pub</a>) - very cool idea of cGAN application. You can also look for U-NET type networks (encoder-decoder). In my opinion autoencoder will not work for this application.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-17T09:14:13.023Z
Answers.Text: Thank you for your answer. Indeed the pix2pix GAN would be useful for my project and is actually my current project. I've never heard about IMO autoencoder, what is it ?
  Are you also aware explainable machine learning methods for this kind of neural network? It would be very useful for my project to see the working procedure of my neural network.
Answers.UserDisplayName: Gaetan Percebois
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-17T10:19:32.310Z
Answers.Text: Sorry, IMO = in my opinion :) Just edited the answer. Take a look at [this](https://www.tensorflow.org/tutorials/generative/pix2pix) tutorial for tensorflow.
Answers.UserDisplayName: Slawomir Orlowski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T15:22:07.910Z
Answers.LastActivityDate: 2022-03-17T10:10:47.063Z
Answers.LastEditDate: 2022-03-17T10:10:47.063Z
Answers.UserDisplayName: Slawomir Orlowski
Body: <p>I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story short, the neural network connects those pairs of images that are very different but related by physical law. I have already something working, but I have a hard time trying to find similar kinds of application of neural networks. Most of the time an autoencoder connect images that are very similar (denoising for example). My two questions are the following:</p>
  <ol>
  <li><p>Are you aware of other similar application of autoencoder?</p>
  </li>
  <li><p>What methods can I use to explain the working procedure of my neural networks? (In the literature the methods concerne mainly classification tasks.)</p>
  </li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-16T11:51:04.347Z
Comments.Text: It's not really an autoencoder if the output is expected to be different to the input. What you appear to have is an image-to-image function, and I guess the only "auto" part is that you have some other reliable method of calculating the output from the input as opposed to measuring it for a dataset? The architecture can be very similar to an autoencoder of course. Maybe look up image encoder/decoder architectures, because there are many similar applications IMO.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-17T09:22:20.107Z
Comments.Text: Thank you for your answer. Indeed, I am looking for an image-to-image function. It might be a stupid question, but I don't know the difference the architecture of an autoencoder and an encoder-decoder. Is there a well defined difference between them?
Comments.UserDisplayName: Gaetan Percebois
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-17T09:30:15.590Z
Comments.Text: There isn't really a clear architecture difference. The difference is the training process and the eventual role that the network components will take. You *may* prefer an architecture without a bottleneck in your case, making it less like an autoencoder (because there will not be a compressed "embedding" vector in the middle layer). But that depends on how many degrees of freedom are in the input
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-17T09:43:41.367Z
Comments.Text: I see what mean, thank you for the clear explanation. I will take this difference into account. Are you aware of application of such encoder decoder architecture? Because so far I didn't find any image to image neural network in the case of the input images is very different of the output image
Comments.UserDisplayName: Gaetan Percebois
Tags: <autoencoders><explainable-ai>
Title: What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?
UserDisplayName: Gaetan Percebois
ThreadText: # What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?
  
  (Asked by: username_0 on 2022-03-16)
  
  I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story short, the neural network connects those pairs of images that are very different but related by physical law. I have already something working, but I have a hard time trying to find similar kinds of application of neural networks. Most of the time an autoencoder connect images that are very similar (denoising for example). My two questions are the following:
  
  1. Are you aware of other similar application of autoencoder?
  2. What methods can I use to explain the working procedure of my neural networks? (In the literature the methods concerne mainly classification tasks.)
  
  **Tags:** autoencoders, explainable-ai
  > **Comments:**
  > **username_1**: It's not really an autoencoder if the output is expected to be different to the input. What you appear to have is an image-to-image function, and I guess the only "auto" part is that you have some other reliable method of calculating the output from the input as opposed to measuring it for a dataset? The architecture can be very similar to an autoencoder of course. Maybe look up image encoder/decoder architectures, because there are many similar applications IMO.  
  > **username_0**: Thank you for your answer. Indeed, I am looking for an image-to-image function. It might be a stupid question, but I don't know the difference the architecture of an autoencoder and an encoder-decoder. Is there a well defined difference between them?  
  > **username_1**: There isn't really a clear architecture difference. The difference is the training process and the eventual role that the network components will take. You *may* prefer an architecture without a bottleneck in your case, making it less like an autoencoder (because there will not be a compressed "embedding" vector in the middle layer). But that depends on how many degrees of freedom are in the input  
  > **username_0**: I see what mean, thank you for the clear explanation. I will take this difference into account. Are you aware of application of such encoder decoder architecture? Because so far I didn't find any image to image neural network in the case of the input images is very different of the output image  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-03-16. Score: 1
  
  If I understand correctly, you are looking for the image-to-image translation method. For example take a look at pix2pix GAN ([pub](https://arxiv.org/abs/1611.07004)) - very cool idea of cGAN application. You can also look for U-NET type networks (encoder-decoder). In my opinion autoencoder will not work for this application.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. Indeed the pix2pix GAN would be useful for my project and is actually my current project. I've never heard about IMO autoencoder, what is it ? Are you also aware explainable machine learning methods for this kind of neural network? It would be very useful for my project to see the working procedure of my neural network.  
  > **username_2**: Sorry, IMO = in my opinion :) Just edited the answer. Take a look at [this](https://www.tensorflow.org/tutorials/generative/pix2pix) tutorial for tensorflow.  
  
  ---
  
  