Answers.Body: <blockquote>
    <p>I can not seem to grasp which specs make an encoder better than another one  </p>
  </blockquote>
  
  <p>In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent variables that explain the variations observed in the training set.  </p>
  
  <p>The autoencoder family (Variational, Denoising, Contrastive, Sparse) try to approximate p(x) so we have a performance metric to tell us how our model is doing.  e.g. (negative log likelihood of p(x))</p>
  
  <blockquote>
    <p>lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats,   </p>
  </blockquote>
  
  <p>If you use some autoencoder model to learn the distribution of cats, you could use the encoder part to further augmented with a linear classifier to discriminate between cats and other categories Therefore you have <strong>an intrinsic task</strong> (learn a good representation of the data distribution) and <strong>an extrinsic task</strong> (learn to classify cat vs not a cat). So you could do a hyper-parameter search for the model that best suits your problem by measuring its accuracy on the extrinsic task.</p>
  
  <p>Side note:<br>
  GAN (Generative Adversarial Network) is a generative model, it provides some way of interacting less directly with this p(x) by drawing samples from it starting without any input. thus the situation here is different.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T07:28:36.660Z
Answers.Text: the link mentioned above is simply explaining the naive autoencoder, the bottom line, data in high dimensional space always lie in a lower dimensional space called manifold, where the probability concentrates and decrease abruptly  as you go far away from the manifold, of course, the manifold could be composed of a set of non-connected manifolds (modes), for more explanation see ch14 of http://www.deeplearningbook.org/
Answers.UserDisplayName: Fadi Bakoura
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:20:22.063Z
Answers.Text: So if I understood correctly, for an autoencoder, Id calculate the probability of a function h given data x that best describes x? Lets say, my function that describes x is x^3 + 1, so that means I would go through each vector of x and calculate its probability falling under that function?
Answers.UserDisplayName: Gabriele
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:28:58.580Z
Answers.Text: If the data distribution is concentrated on x^3+1 with some variance, the model will try to put more probability on points that are similar to the training data and puts a lower probability on points that don't resemble the data distribution based on the maximum likelihood principle.
Answers.UserDisplayName: Fadi Bakoura
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:42:03.023Z
Answers.Text: What if I do not know the probability? What if I start with a function x +1 even though the real h is x^3 + 1
Answers.UserDisplayName: Gabriele
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:45:54.253Z
Answers.Text: You don't need to know the probability, you just have training data, where they have some internal structure, you just exploit it by letting the model learns P(x)
Answers.UserDisplayName: Fadi Bakoura
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:53:06.920Z
Answers.Text: Can you give an example to me?
Answers.UserDisplayName: Gabriele
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-21T08:53:56.993Z
Answers.Text: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/77786/discussion-between-gabriele-and-fadi-bakoura).
Answers.UserDisplayName: Gabriele
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-18T12:08:05.157Z
Answers.LastActivityDate: 2018-05-18T12:08:05.157Z
Answers.UserDisplayName: Fadi Bakoura
Body: <p>I am trying to assess an encoder in my autoencoder. I can not seem to grasp which specs make an encoder better than other one in, lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats, so that when I provide a picture of a bird, my autoencoder would tell me that it is not a picture of a cat. I am trying to understand what exact specs make my encoder (and decoder) better? I understand it is all about chosen weights but is it possible to be more specific? </p>
  
Tags: <autoencoders>
Title: What are good parameters of an encoder?
UserDisplayName: Gabriele
ThreadText: # What are good parameters of an encoder?
  
  (Asked by: username_0 on 2018-05-18)
  
  I am trying to assess an encoder in my autoencoder. I can not seem to grasp which specs make an encoder better than other one in, lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats, so that when I provide a picture of a bird, my autoencoder would tell me that it is not a picture of a cat. I am trying to understand what exact specs make my encoder (and decoder) better? I understand it is all about chosen weights but is it possible to be more specific?
  
  **Tags:** autoencoders
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-18. Score: 2 | Accepted answer
  
  > I can not seem to grasp which specs make an encoder better than another one
  
  In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent variables that explain the variations observed in the training set.
  
  The autoencoder family (Variational, Denoising, Contrastive, Sparse) try to approximate p(x) so we have a performance metric to tell us how our model is doing. e.g. (negative log likelihood of p(x))
  
  > lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats,
  
  If you use some autoencoder model to learn the distribution of cats, you could use the encoder part to further augmented with a linear classifier to discriminate between cats and other categories Therefore you have **an intrinsic task** (learn a good representation of the data distribution) and **an extrinsic task** (learn to classify cat vs not a cat). So you could do a hyper-parameter search for the model that best suits your problem by measuring its accuracy on the extrinsic task.
  
  Side note:  
  GAN (Generative Adversarial Network) is a generative model, it provides some way of interacting less directly with this p(x) by drawing samples from it starting without any input. thus the situation here is different.
  
  > **Comments:**
  > **username_1**: the link mentioned above is simply explaining the naive autoencoder, the bottom line, data in high dimensional space always lie in a lower dimensional space called manifold, where the probability concentrates and decrease abruptly  as you go far away from the manifold, of course, the manifold could be composed of a set of non-connected manifolds (modes), for more explanation see ch14 of http://www.deeplearningbook.org/  
  > **username_0**: So if I understood correctly, for an autoencoder, Id calculate the probability of a function h given data x that best describes x? Lets say, my function that describes x is x^3 + 1, so that means I would go through each vector of x and calculate its probability falling under that function?  
  > **username_1**: If the data distribution is concentrated on x^3+1 with some variance, the model will try to put more probability on points that are similar to the training data and puts a lower probability on points that don't resemble the data distribution based on the maximum likelihood principle.  
  > **username_0**: What if I do not know the probability? What if I start with a function x +1 even though the real h is x^3 + 1  
  > **username_1**: You don't need to know the probability, you just have training data, where they have some internal structure, you just exploit it by letting the model learns P(x)  
  > **username_0**: Can you give an example to me?  
  > **username_0**: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/77786/discussion-between-username_0-and-username_1-username_1).  
  
  ---
  
  