Answers.Body: <p>In short, repetition with feedback.</p>
  <p>You are correct that machine learning (ML) models such as neural networks work with fixed dimensions for input and output. There are a few different ways to work around this when desired input and output are more variable. The most common approaches are:</p>
  <ul>
  <li><p><strong>Padding:</strong> Give the ML model capacity to cope with the largest expected dimensions then pad inputs and filter outputs as necessary to match logical requirements. For example, this might be used for an image classifier where the input image varies in size and shape.</p>
  </li>
  <li><p><strong>Recurrent models:</strong> Add an internal state to the ML model and use it to pass data along with each input, in order to work with <em>sequences</em> of identical, related inputs or outputs. This is a preferred architecture for natural language processing (NLP) tasks, where LSTMs, GRUs, and transformer networks are common choices.</p>
  </li>
  </ul>
  <p>A recurrent model relies on the fact that each input and output is the same kind of thing, at a different point in the sequence. The internal state of the model is used to combine information between points in the sequence so that for example a word in position three in the input has an impact on the choice of the word at position seven of the output.</p>
  <p>Generative recurrent models often use their own output (or a sample based on the probabilities expressed in the output) as the next step's input.</p>
  <p>It is well worth reading this blog for an introduction and some examples: <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" rel="nofollow noreferrer">The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-01T08:14:31.660Z
Answers.Text: This answer could probably do with an update to explain how transformers do this. I am not 100% sure, but it probably involves manipulating the input based on previous output, similar to, but not the same as RNNs.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-24T08:12:36.020Z
Answers.LastActivityDate: 2021-12-17T00:15:43.720Z
Answers.LastEditDate: 2021-12-17T00:15:43.720Z
Answers.UserDisplayName: Neil Slater
Body: <p>Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable length. How is this possible?</p>
  
Tags: <machine-learning><natural-language-processing><recurrent-neural-networks><transformer><natural-language-generation>
Title: How are certain machine learning models able to produce variable-length outputs given variable-length inputs?
UserDisplayName: Python Test
ThreadText: # How are certain machine learning models able to produce variable-length outputs given variable-length inputs?
  
  (Asked by: username_0 on 2021-02-24)
  
  Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable length. How is this possible?
  
  **Tags:** machine-learning, natural-language-processing, recurrent-neural-networks, transformer, natural-language-generation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-02-24. Score: 4 | Accepted answer
  
  In short, repetition with feedback.
  
  You are correct that machine learning (ML) models such as neural networks work with fixed dimensions for input and output. There are a few different ways to work around this when desired input and output are more variable. The most common approaches are:
  
  * **Padding:** Give the ML model capacity to cope with the largest expected dimensions then pad inputs and filter outputs as necessary to match logical requirements. For example, this might be used for an image classifier where the input image varies in size and shape.
  * **Recurrent models:** Add an internal state to the ML model and use it to pass data along with each input, in order to work with *sequences* of identical, related inputs or outputs. This is a preferred architecture for natural language processing (NLP) tasks, where LSTMs, GRUs, and transformer networks are common choices.
  
  A recurrent model relies on the fact that each input and output is the same kind of thing, at a different point in the sequence. The internal state of the model is used to combine information between points in the sequence so that for example a word in position three in the input has an impact on the choice of the word at position seven of the output.
  
  Generative recurrent models often use their own output (or a sample based on the probabilities expressed in the output) as the next step's input.
  
  It is well worth reading this blog for an introduction and some examples: [The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
  
  > **Comments:**
  > **username_1**: This answer could probably do with an update to explain how transformers do this. I am not 100% sure, but it probably involves manipulating the input based on previous output, similar to, but not the same as RNNs.  
  
  ---
  
  