Answers.Body: <p>One such paper I know of and which I implemented is <a href="https://arxiv.org/abs/1507.02672" rel="nofollow noreferrer">Semi-Supervised Learning using Ladder Networks</a> . I quote here their description of the model:</p>
  
  <blockquote>
    <p>Our approach follows Valpola (2015), who proposed a Ladder network where the auxiliary task is
    to denoise representations at every level of the model.  The model structure is an autoencoder with
    skip connections from the encoder to decoder and the learning task is similar to that in denoising
    autoencoders but applied to every layer, not just the inputs. The skip connections relieve the pressure
    to represent details in the higher layers of the model because,  through the skip connections, the decoder can recover any details discarded by the encoder.</p>
  </blockquote>
  
  <p>For further explanations on the architecture check <a href="https://arxiv.org/abs/1511.06430" rel="nofollow noreferrer">Deconstructing the Ladder Network Architecture</a> by Yoshua Bengio.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-31T09:32:33.213Z
Answers.LastActivityDate: 2017-05-31T09:32:33.213Z
Answers.UserDisplayName: AlexGuevara
Answers.Body: <p>I recall reading papers about such systems, if I understand you correctly, but can't recall the titles at the moment.</p>
  
  <p>The idea was to use character-based generative RNNs, train them on sequences encoded like "datadatadatadata|answer", and then when feeding in "otherdatadata|" then it would continue to generate some kind of expected answer. </p>
  
  <p>But, as far as I recall, that was just a neat illustration since if you have the data to do something supervised, then you'd get better results by conventional methods.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-02T11:48:57.190Z
Answers.LastActivityDate: 2016-09-02T11:48:57.190Z
Answers.UserDisplayName: Peteris
Body: <p>Can autoencoders be used for supervised learning <em>without adding an output layer</em>? Can we simply feed it with a concatenated input-output vector for training, and reconstruct the output part from the input part when doing inference? The output part would be treated as missing values during inference and some imputation would be applied.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-04T20:19:57.650Z
Comments.Text: I don't quite understand. If you train it with input-output vectors, you'll also need output vectors while doing inference to feed it to the network. What are you going to do about that?
Comments.UserDisplayName: Didami
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-04T20:48:05.673Z
Comments.Text: No, they would be treated as missing values and imputed in some way. The autoencoder would then try to reconstruct it (multiple iterations may be necessary,). The question is precisely about the feasibility of this idea. I'll edit to clarify.
Comments.UserDisplayName: rcpinto
Tags: <neural-networks><autoencoders><supervised-learning>
Title: Can autoencoders be used for supervised learning?
UserDisplayName: rcpinto
ThreadText: # Can autoencoders be used for supervised learning?
  
  (Asked by: username_0 on 2016-08-02)
  
  Can autoencoders be used for supervised learning *without adding an output layer*? Can we simply feed it with a concatenated input-output vector for training, and reconstruct the output part from the input part when doing inference? The output part would be treated as missing values during inference and some imputation would be applied.
  
  **Tags:** neural-networks, autoencoders, supervised-learning
  > **Comments:**
  > **username_1**: I don't quite understand. If you train it with input-output vectors, you'll also need output vectors while doing inference to feed it to the network. What are you going to do about that?  
  > **username_0**: No, they would be treated as missing values and imputed in some way. The autoencoder would then try to reconstruct it (multiple iterations may be necessary,). The question is precisely about the feasibility of this idea. I'll edit to clarify.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-05-31. Score: 2
  
  One such paper I know of and which I implemented is [Semi-Supervised Learning using Ladder Networks](https://arxiv.org/abs/1507.02672) . I quote here their description of the model:
  
  > Our approach follows Valpola (2015), who proposed a Ladder network where the auxiliary task is
  > to denoise representations at every level of the model. The model structure is an autoencoder with
  > skip connections from the encoder to decoder and the learning task is similar to that in denoising
  > autoencoders but applied to every layer, not just the inputs. The skip connections relieve the pressure
  > to represent details in the higher layers of the model because, through the skip connections, the decoder can recover any details discarded by the encoder.
  
  For further explanations on the architecture check [Deconstructing the Ladder Network Architecture](https://arxiv.org/abs/1511.06430) by Yoshua Bengio.
  
  
  ---
  
  ### Answer by username_3 on 2016-09-02. Score: 1
  
  I recall reading papers about such systems, if I understand you correctly, but can't recall the titles at the moment.
  
  The idea was to use character-based generative RNNs, train them on sequences encoded like "datadatadatadata|answer", and then when feeding in "otherdatadata|" then it would continue to generate some kind of expected answer.
  
  But, as far as I recall, that was just a neat illustration since if you have the data to do something supervised, then you'd get better results by conventional methods.
  
  
  ---
  
  