Body: <p>I'm wondering what are the different choices of parametrizations available for the decoder in a variational autoencoder. If the data is discrete, you can just output probabilities for each class, so that's straightforward. If the data is <strong>a)</strong> continuous and not bounded, then the only parametrization I know of is the (multivariate) normal distribution. If the data is <strong>b)</strong> continuous and bounded, again the only parametrization I know of is a (multivariate) normal distribution, mapped to [0,1] using a sigmoid function.</p>
  <p>Are there other formulations that are commonly used? If there are, what are the advantages and any disadvantages compared to using a normal distribution?</p>
  <p>To ask the question in a different way: the normal distribution seems like the &quot;go to&quot; choice in problems with continuous output since it's simple and allows gradients through the reparametrization trick. Are there other reparametrizable continuous distributions which are also commonly used in machine learning? I know the exponential distribution (and related distributions such as weibull, gamma) are reparametrizable, but would you ever use those instead of a normal distribution to model continuous data?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-18T14:44:06.450Z
Comments.Text: In first paragraph, you are asking about the data distribution "for the decoder", i.e. `p(x|z)` while in the last paragraph, you are asking a very different question about `p(z|x)` and its reparameterizability. Kindly clarify which one you want to know, or you want both ?
Comments.UserDisplayName: ayandas
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-18T16:39:18.427Z
Comments.Text: The question is about when (if ever) you would ever use something other than a normal distribution to model continuous data.
Comments.UserDisplayName: Taw
Tags: <generative-model><variational-autoencoder>
Title: Types of decoder parametrizations in VAE for continuous data
UserDisplayName: Taw
ThreadText: # Types of decoder parametrizations in VAE for continuous data
  
  (Asked by: username_0 on 2021-08-17)
  
  I'm wondering what are the different choices of parametrizations available for the decoder in a variational autoencoder. If the data is discrete, you can just output probabilities for each class, so that's straightforward. If the data is **a)** continuous and not bounded, then the only parametrization I know of is the (multivariate) normal distribution. If the data is **b)** continuous and bounded, again the only parametrization I know of is a (multivariate) normal distribution, mapped to [0,1] using a sigmoid function.
  
  Are there other formulations that are commonly used? If there are, what are the advantages and any disadvantages compared to using a normal distribution?
  
  To ask the question in a different way: the normal distribution seems like the "go to" choice in problems with continuous output since it's simple and allows gradients through the reparametrization trick. Are there other reparametrizable continuous distributions which are also commonly used in machine learning? I know the exponential distribution (and related distributions such as weibull, gamma) are reparametrizable, but would you ever use those instead of a normal distribution to model continuous data?
  
  **Tags:** generative-model, variational-autoencoder
  
  > **Comments:**
  > **username_1**: In first paragraph, you are asking about the data distribution "for the decoder", i.e. `p(x|z)` while in the last paragraph, you are asking a very different question about `p(z|x)` and its reparameterizability. Kindly clarify which one you want to know, or you want both ?  
  > **username_0**: The question is about when (if ever) you would ever use something other than a normal distribution to model continuous data.  
  
  ---
  
  ## 0 answers
  
  