Answers.Body: <p>I think that, in the case of a (deterministic) <a href="https://www.tensorflow.org/tutorials/generative/autoencoder" rel="nofollow noreferrer">auto-encoder</a> that only compresses the data, which is trained with MSE, it's <strong>an assumption</strong> or <strong>convention</strong> that the encoder is just that part of the neural network that <strong>compresses the data</strong> into a more compact representation/vector.</p>
  <p>However, I understand your concern, because, if you train the encoder and decoder jointly, one may think that the decoder is also responsible for <em>encoding</em>, which I assume you define as the transformation of the input into something else, before reconstructing it. I don't know to what extent one may entertain this idea, but, honestly, I would just stick to the convention of considering the encoder the part that compresses the data because, in my mind at least, encoding usually means to compress, and that's what the encoder does. If you define encoding as any transformation, then any neural network, or any layer, or, even more generally, any function, can be considered an encoder, but it's not what people (in ML) usually refer to when they mention an encoder.</p>
  <p>In the case of the <a href="https://arxiv.org/pdf/1312.6114.pdf" rel="nofollow noreferrer">VAE</a>, the situation is even clearer, I would say, because the VAE was formulated to model a specific graphical model, where <span class="math-container">$x$</span>, the (original/reconstructed) input depends on a latent variable <span class="math-container">$z$</span>, although, even in this case, the encoder and decoder are trained jointly with the ELBO, which contains 2 terms, one for the encoder and the other for the decoder. It's true that the optimization of this objective takes into account both terms, but, again, the convention is to consider the encoder the part that produces the latent vector.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-10T09:08:24.617Z
Answers.LastActivityDate: 2022-07-10T09:13:49.490Z
Answers.LastEditDate: 2022-07-10T09:13:49.490Z
Answers.UserDisplayName: nbro
Body: <p>Consider a simple Autoencoder neural net:</p>
  <pre><code>from torch import nn
  
  class AE(nn.Module):
      def __init__(self, x_dim, z_dim, h_dim=42):
          super().__init__()
          self.encoder = nn.Sequential(
              nn.Linear(dim_in, dim_h),
              nn.ReLU(),
              nn.Linear(dim_h, z_dim)
          )
          self.decoder = nn.Sequential(
              nn.Linear(z_dim, h_dim),
              nn.ReLU(),
              nn.Linear(h_dim, x_dim),
          )
  
      def forward(self, x):
          z = self.encoder(x)
          x = self.decoder(z)
          return x
  </code></pre>
  <p>In popular literature, it is generally implied that the output of <code>AE.encoder</code> is <em>solely</em> responsible for encoding whereas <code>AE.decoder</code> is <em>solely</em> responsible for the decoding.
  Why is that?</p>
  <p>If we consider that encoding is a more complex task than decoding, there is no actual guarantee that the network won't use the first three layers for encoding and only the last for decoding (or vice versa). This might especially be the case if we consider asymmetrical autoencoder architectures.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-09T16:19:49.087Z
Comments.Text: The encoder ends at the layer where the representation is most compact, i.e. at the bottleneck, because it wouldn't make sense to have a narrower bottleneck *within* the encoder or *within* the decoder.
Comments.UserDisplayName: Martino
Tags: <neural-networks><machine-learning><definitions><autoencoders>
Title: Autoencoders: Where does the encoder end and the decoder begin?
UserDisplayName: John Titor
ThreadText: # Autoencoders: Where does the encoder end and the decoder begin?
  
  (Asked by: username_0 on 2022-07-09)
  
  Consider a simple Autoencoder neural net:
  from torch import nn
  
  class AE(nn.Module):
      def __init__(self, x_dim, z_dim, h_dim=42):
          super().__init__()
          self.encoder = nn.Sequential(
              nn.Linear(dim_in, dim_h),
              nn.ReLU(),
              nn.Linear(dim_h, z_dim)
          )
          self.decoder = nn.Sequential(
              nn.Linear(z_dim, h_dim),
              nn.ReLU(),
              nn.Linear(h_dim, x_dim),
          )
  
      def forward(self, x):
          z = self.encoder(x)
          x = self.decoder(z)
          return x
  
  In popular literature, it is generally implied that the output of AE.encoder is solely responsible for encoding whereas AE.decoder is solely responsible for the decoding.
  Why is that?
  If we consider that encoding is a more complex task than decoding, there is no actual guarantee that the network won't use the first three layers for encoding and only the last for decoding (or vice versa). This might especially be the case if we consider asymmetrical autoencoder architectures.
  
  **Tags:** neural-networks, machine-learning, definitions, autoencoders
  > **Comments:**
  > **username_1**: The encoder ends at the layer where the representation is most compact, i.e. at the bottleneck, because it wouldn't make sense to have a narrower bottleneck *within* the encoder or *within* the decoder.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-07-10. Score: 1
  
  I think that, in the case of a (deterministic) [auto-encoder](https://www.tensorflow.org/tutorials/generative/autoencoder) that only compresses the data, which is trained with MSE, it's **an assumption** or **convention** that the encoder is just that part of the neural network that **compresses the data** into a more compact representation/vector.
  
  However, I understand your concern, because, if you train the encoder and decoder jointly, one may think that the decoder is also responsible for *encoding*, which I assume you define as the transformation of the input into something else, before reconstructing it. I don't know to what extent one may entertain this idea, but, honestly, I would just stick to the convention of considering the encoder the part that compresses the data because, in my mind at least, encoding usually means to compress, and that's what the encoder does. If you define encoding as any transformation, then any neural network, or any layer, or, even more generally, any function, can be considered an encoder, but it's not what people (in ML) usually refer to when they mention an encoder.
  
  In the case of the [VAE](https://arxiv.org/pdf/1312.6114.pdf), the situation is even clearer, I would say, because the VAE was formulated to model a specific graphical model, where $x$, the (original/reconstructed) input depends on a latent variable $z$, although, even in this case, the encoder and decoder are trained jointly with the ELBO, which contains 2 terms, one for the encoder and the other for the decoder. It's true that the optimization of this objective takes into account both terms, but, again, the convention is to consider the encoder the part that produces the latent vector.
  
  
  ---
  
  