Answers.Body: <p>Usually, in natural language processing (NLP), they are using Sequence to Sequence Learning (Seq2Seq) with Neural Networks, such as Recurrent Neural Networks or more recently the Transformer (you can find two very good papers <a href="https://arxiv.org/abs/1409.3215" rel="nofollow noreferrer">here</a>, and <a href="https://arxiv.org/abs/1706.03762" rel="nofollow noreferrer">here</a>).</p>
  <p>During <em>training</em>, to ensure the same size of the input and output they can just search for the longest sentence they have in the dataset or select a number that is high enough and pad all the other sentences with 0. In addition, they add a stop token where the sentence ends, so that the model is aware of this. When <em>decoding (inference)</em>, the decoder is going to predict one word at a time until it predicts the stop token, which signals that the translation is done.</p>
  <p>If you're interested to see an actual implementation, I would recommend looking at <a href="https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html" rel="nofollow noreferrer">this</a> tutorial which does a good job at explaining the code and how it works.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-10T10:51:34.620Z
Answers.Text: Transformers have surpassed RNNs (even LSTMs) in some NLP tasks (including translation), and they are now used by Google Translate (check [this](https://ai.stackexchange.com/a/23042/2444)), so you probably should mention them in this answer.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-10T09:39:18.860Z
Answers.LastActivityDate: 2020-11-10T11:38:28.067Z
Answers.LastEditDate: 2020-11-10T11:38:28.067Z
Answers.UserDisplayName: razvanc92
Body: <p>According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?</p>
  
Tags: <natural-language-processing><recurrent-neural-networks><machine-translation><google-translate><seq2seq>
Title: How is Google Translate able to convert texts of different lengths?
UserDisplayName: Gavin
ThreadText: # How is Google Translate able to convert texts of different lengths?
  
  (Asked by: username_0 on 2020-11-10)
  
  According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
  
  **Tags:** natural-language-processing, recurrent-neural-networks, machine-translation, google-translate, seq2seq
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-11-10. Score: 4
  
  Usually, in natural language processing (NLP), they are using Sequence to Sequence Learning (Seq2Seq) with Neural Networks, such as Recurrent Neural Networks or more recently the Transformer (you can find two very good papers [here](https://arxiv.org/abs/1409.3215), and [here](https://arxiv.org/abs/1706.03762)).
  
  During *training*, to ensure the same size of the input and output they can just search for the longest sentence they have in the dataset or select a number that is high enough and pad all the other sentences with 0. In addition, they add a stop token where the sentence ends, so that the model is aware of this. When *decoding (inference)*, the decoder is going to predict one word at a time until it predicts the stop token, which signals that the translation is done.
  
  If you're interested to see an actual implementation, I would recommend looking at [this](https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html) tutorial which does a good job at explaining the code and how it works.
  
  > **Comments:**
  > **username_2**: Transformers have surpassed RNNs (even LSTMs) in some NLP tasks (including translation), and they are now used by Google Translate (check [this](https://ai.stackexchange.com/a/23042/2444)), so you probably should mention them in this answer.  
  
  ---
  
  