Answers.Body: <p>You will find that the nomenclature in this field is not particularly rigorous. One of the main reasons I visit this site is to keep up with the new names professors (and others) are using for existing things.</p>
  <p>That said, an autoencoder is comprised of an encoder, a latent space (some type of information choke point), and a decoder. Based on this and given the options you mention, a strong argument can be made that it is an encoder-decoder model, however it is not a sequence-to-sequence model at all. It is simply attempting to reconstruct the input, not predict a future value, token, or anything else.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-10T21:11:53.113Z
Answers.Text: Based on your explanation, you're indicating that an encoder-decoder model is **not** a sequence-to-sequence model. However, when you consider for instance machine translation which is a sequence-to-sequence task, this is often modelled using encoder-decoder models.
Answers.UserDisplayName: KurtMica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-10T22:18:30.747Z
Answers.Text: LSTMs and RNNs tend to be sequence to sequence encoder-decoder models. My point is that Autoencoders *are* encoder-decoder models but they are not used for sequence to sequence tasks.
Answers.UserDisplayName: David Hoelzer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-03T21:00:21.700Z
Answers.LastActivityDate: 2024-10-03T21:00:21.700Z
Answers.UserDisplayName: David Hoelzer
Body: <p>I'm writing up about different model architectures used in NLP, namely encoder-only models, encoder-decoder-only models, and have come across what seems to be a naming inconsistency. For decoder-only models it seems that they can be referred to as autoregressive models since they autoregressively predict the next token and encoder-decoder models can be referred to as sequence-to-sequence since they map a sequence of tokens to a different sequence of tokens.</p>
  <p>However, I found various sources conflicting each other as to what an autoencoder is. While many sources claim that encoder-only models are autoencoders, others also refer to encoder-decoder models as being autoencoders.</p>
  
Tags: <natural-language-processing><autoencoders><language-model><encoder-decoder><decoder>
Title: Is an autoencoder model encoder-only or encoder-decoder?
UserDisplayName: KurtMica
ThreadText: # Is an autoencoder model encoder-only or encoder-decoder?
  
  (Asked by: username_0 on 2024-10-03)
  
  I'm writing up about different model architectures used in NLP, namely encoder-only models, encoder-decoder-only models, and have come across what seems to be a naming inconsistency. For decoder-only models it seems that they can be referred to as autoregressive models since they autoregressively predict the next token and encoder-decoder models can be referred to as sequence-to-sequence since they map a sequence of tokens to a different sequence of tokens.
  
  However, I found various sources conflicting each other as to what an autoencoder is. While many sources claim that encoder-only models are autoencoders, others also refer to encoder-decoder models as being autoencoders.
  
  **Tags:** natural-language-processing, autoencoders, language-model, encoder-decoder, decoder
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-10-03. Score: 2
  
  You will find that the nomenclature in this field is not particularly rigorous. One of the main reasons I visit this site is to keep up with the new names professors (and others) are using for existing things.
  
  That said, an autoencoder is comprised of an encoder, a latent space (some type of information choke point), and a decoder. Based on this and given the options you mention, a strong argument can be made that it is an encoder-decoder model, however it is not a sequence-to-sequence model at all. It is simply attempting to reconstruct the input, not predict a future value, token, or anything else.
  
  > **Comments:**
  > **username_0**: Based on your explanation, you're indicating that an encoder-decoder model is **not** a sequence-to-sequence model. However, when you consider for instance machine translation which is a sequence-to-sequence task, this is often modelled using encoder-decoder models.  
  > **username_1**: LSTMs and RNNs tend to be sequence to sequence encoder-decoder models. My point is that Autoencoders *are* encoder-decoder models but they are not used for sequence to sequence tasks.  
  
  ---
  
  