Answers.Body: <p>You need more training images. Far more, at least a few hundred, with variations. The latent space has no meaningful form to it when you train with just two end points. The decoder will have no examples of shifting forms that it could arrange on the path between typical 1s and typical 7s.</p>
  <p>Also, if there are separate classes within the encoded space, representations in the middle are not guaranteed to be coherent transformations between them. More training examples will help, but some parts of the latent space will be odd. In human face examples you can often observe this when latent  space crosses between glasses and no glasses, the transition can fall apart.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-23T17:51:04.903Z
Answers.Text: thank you, I think I understand how morphing works now. Although, I still can't wrap my head around how smile replacement works (where a non-smiling face is replaced by a smiling face, but the rest of the face is unchanged). Is this also done through latent space pathfinding? But how to keep the rest of the face unchanged while changing only the smile around the mouth?
Answers.UserDisplayName: James
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-23T18:03:58.203Z
Answers.Text: @James that would be a different question, but the principle is to try and extract the "smile vector" by using all the labelled examples. It's rarely perfect, and often other image features change too
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-23T17:25:55.283Z
Answers.LastActivityDate: 2024-03-23T17:25:55.283Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have a variational convolutional autoencoder that has trained on 2 images and outputs a linear interpolation (inserted at the bottleneck stage) between those 2 input images.</p>
  <p><a href="https://i.sstatic.net/zPOw4.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/zPOw4.gif" alt="enter image description here" /></a></p>
  <p>However, the result looks (rather dissapointingly) like some Powerpoint cross-fade effect.</p>
  <p>What would be needed to obtain a more novel transition effect, for instance limbs moving from '1' and seamlessly reshaping into '7'?</p>
  
Tags: <convolutional-neural-networks><variational-autoencoder><u-net>
Title: How to perform latent space Interpolation between two images?
UserDisplayName: James
ThreadText: # How to perform latent space Interpolation between two images?
  
  (Asked by: username_0 on 2024-03-23)
  
  I have a variational convolutional autoencoder that has trained on 2 images and outputs a linear interpolation (inserted at the bottleneck stage) between those 2 input images.
  
  [![enter image description here](https://i.sstatic.net/zPOw4.gif)](https://i.sstatic.net/zPOw4.gif)
  
  However, the result looks (rather dissapointingly) like some Powerpoint cross-fade effect.
  
  What would be needed to obtain a more novel transition effect, for instance limbs moving from '1' and seamlessly reshaping into '7'?
  
  **Tags:** convolutional-neural-networks, variational-autoencoder, u-net
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-03-23. Score: 3 | Accepted answer
  
  You need more training images. Far more, at least a few hundred, with variations. The latent space has no meaningful form to it when you train with just two end points. The decoder will have no examples of shifting forms that it could arrange on the path between typical 1s and typical 7s.
  
  Also, if there are separate classes within the encoded space, representations in the middle are not guaranteed to be coherent transformations between them. More training examples will help, but some parts of the latent space will be odd. In human face examples you can often observe this when latent space crosses between glasses and no glasses, the transition can fall apart.
  
  > **Comments:**
  > **username_0**: thank you, I think I understand how morphing works now. Although, I still can't wrap my head around how smile replacement works (where a non-smiling face is replaced by a smiling face, but the rest of the face is unchanged). Is this also done through latent space pathfinding? But how to keep the rest of the face unchanged while changing only the smile around the mouth?  
  > **username_1**: @James that would be a different question, but the principle is to try and extract the "smile vector" by using all the labelled examples. It's rarely perfect, and often other image features change too  
  
  ---
  
  