Answers.Body: <p>Since you have already tried U-Net. You may look into Siamese Networks (with CNNs for images), they are very well known for computing similarity via deep learning. This is a central idea and can be performed with both text and images. As a tip, you may be able to leverage a lot of architecture from U-Net to Siamese. </p>
  
  <p>Hope it helps, Some useful links to start with :</p>
  
  <ol>
  <li><a href="https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04" rel="nofollow noreferrer">https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04</a></li>
  <li><a href="https://www.aclweb.org/anthology/W16-1617.pdf" rel="nofollow noreferrer">https://www.aclweb.org/anthology/W16-1617.pdf</a></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-27T07:59:42.620Z
Answers.Text: Hi, thank you for the tip. However I don't see how I could use the siamese architecture in my problem, I have only one image from which I should output another image, I am not comparing anything
Answers.UserDisplayName: Giulio Ortali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T04:31:47.470Z
Answers.Text: It will be great if you can provide more information about your dataset. (train/test). Another suggestion could convert your U-NET to VAE (variational autoencoder). Using this you can generate more images given an image. Let me know if this helps I can edit the answer above.  Training setup can be of 2 types with VAE (1)unsupervised and (2)supervised. Usually, its known to be trained in unsupervised fashion but in your case supervised could be better. Again, this raises, why this is different from U-NET?- I think because you can control the variation. But we can observe results empirically.
Answers.UserDisplayName: HARSH NILESH PATHAK
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-26T03:15:52.043Z
Answers.LastActivityDate: 2020-01-26T03:15:52.043Z
Answers.UserDisplayName: HARSH NILESH PATHAK
Body: <p>I am working on a problem in which I need to train a neural network to map one or more input images to one or more output images (1 channel for image). Below I report some examples of input&amp;output. In this case I report 1 input and 1 output image, but may need to pass to more inputs and outputs, maybe by encoding this in channels. However, the images are all of this kind, maybe rotated, tralated or changed a bit in shape. (fyi, they are fields defined by fluid dynamics simulations)</p>
  
  <p>I was thinking about CNN, but the standard architecture used for image classification (convolutional layers + fully connected layers) seems not to be the best choice. Instead, I tried using the U-net architecture, composed of compression+decompression convolutional layers. This works quite fine, but maybe there is some other architecture that could be more suited to my problem.</p>
  
  <p>Any suggestion would be appreciated!</p>
  
  <p><a href="https://i.sstatic.net/BE9Ox.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/BE9Ox.png" alt="input image"></a>
  <a href="https://i.sstatic.net/EwXbV.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/EwXbV.png" alt="output image"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-30T10:45:01.307Z
Comments.Text: did you figured out what worked well. I am kind of working on the similar problem. If will be great if you can share any ideas. Thank you
Comments.UserDisplayName: parth modi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-01T11:14:00.920Z
Comments.Text: Something close to UNet has given the best performances
Comments.UserDisplayName: Giulio Ortali
Tags: <convolutional-neural-networks><computer-vision>
Title: Which deep learning models are suitable for image-to-image mapping?
UserDisplayName: Giulio Ortali
ThreadText: # Which deep learning models are suitable for image-to-image mapping?
  
  (Asked by: username_0 on 2020-01-21)
  
  I am working on a problem in which I need to train a neural network to map one or more input images to one or more output images (1 channel for image). Below I report some examples of input&output. In this case I report 1 input and 1 output image, but may need to pass to more inputs and outputs, maybe by encoding this in channels. However, the images are all of this kind, maybe rotated, tralated or changed a bit in shape. (fyi, they are fields defined by fluid dynamics simulations)
  
  I was thinking about CNN, but the standard architecture used for image classification (convolutional layers + fully connected layers) seems not to be the best choice. Instead, I tried using the U-net architecture, composed of compression+decompression convolutional layers. This works quite fine, but maybe there is some other architecture that could be more suited to my problem.
  
  Any suggestion would be appreciated!
  
  [![input image](https://i.sstatic.net/BE9Ox.png)](https://i.sstatic.net/BE9Ox.png)
  [![output image](https://i.sstatic.net/EwXbV.png)](https://i.sstatic.net/EwXbV.png)
  
  **Tags:** convolutional-neural-networks, computer-vision
  > **Comments:**
  > **username_1**: did you figured out what worked well. I am kind of working on the similar problem. If will be great if you can share any ideas. Thank you  
  > **username_0**: Something close to UNet has given the best performances  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-01-26. Score: 2
  
  Since you have already tried U-Net. You may look into Siamese Networks (with CNNs for images), they are very well known for computing similarity via deep learning. This is a central idea and can be performed with both text and images. As a tip, you may be able to leverage a lot of architecture from U-Net to Siamese.
  
  Hope it helps, Some useful links to start with :
  
  1. <https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04>
  2. <https://www.aclweb.org/anthology/W16-1617.pdf>
  
  > **Comments:**
  > **username_0**: Hi, thank you for the tip. However I don't see how I could use the siamese architecture in my problem, I have only one image from which I should output another image, I am not comparing anything  
  > **username_2**: It will be great if you can provide more information about your dataset. (train/test). Another suggestion could convert your U-NET to VAE (variational autoencoder). Using this you can generate more images given an image. Let me know if this helps I can edit the answer above.  Training setup can be of 2 types with VAE (1)unsupervised and (2)supervised. Usually, its known to be trained in unsupervised fashion but in your case supervised could be better. Again, this raises, why this is different from U-NET?- I think because you can control the variation. But we can observe results empirically.  
  
  ---
  
  