Latent Sequence Decompositions
Sequence-to-sequence models rely on a fixed decomposition of the target sequences into a sequence of tokens that may be words, word-pieces or characters. The choice of these tokens and the decomposition of the target sequences into a sequence of tokens is often static, and independent of the input, output data domains. This can potentially lead to a sub-optimal choice of token dictionaries, as the decomposition is not informed by the particular problem being solved. In this paper we present Latent Sequence Decompositions (LSD), a framework in which the decomposition of sequences into constituent tokens is learnt during the training of the model. The decomposition depends both on the input sequence and on the output sequence. In LSD, during training, the model samples decompositions incrementally, from left to right by locally sampling between valid extensions. We experiment with the Wall Street Journal speech recognition task. Our LSD model achieves 12.9% WER compared to a character baseline of 14.8% WER. When combined with a convolutional network on the encoder, we achieve a WER of 9.6%.
Interesting paper which proposes jointly learning automatic segmentation of words to sub words and their acoustic models. Although the training handles the word segmentation as hidden variable which depends also on the acoustic representations, during the decoding only maximum approximation is used. The authors present nice improvements over character based results, however they did not compare results with word segmentation which does not assume the dependency on acoustic. Obviously, only text based segmentation would result in two (but simpler) independent tasks. In order to extract such segmentation several publicly open tools are available and should be cited. Some of those tools can also exploit the unigram probabilities of the words to perform their segmentations. It looks that the improvements come from the longer acoustical units - longer acoustical constraints which could lead to less confused search -, pointing towards full word models. In another way, less tokens are more probable due to less multiplication of probabilities. As a thought experiment for an extreme case: if all the possible segmentations would be possible (mixture of all word fragments, characters, and full-words), would the proposed model use word fragments at all? (WSJ is a closed vocabulary task). It would be good to show that the sub word model could outperform even a full-word model (no segmentation). Your model estimates p(z_t|x,z<t;	heta), but during training both p(z_t|x,y,z<t;	heta) and p(z_t|x,z<t;	heta) are needed. Their connections and calculations, in general Section 2 should be more detailed. The experimental setup should be presented in a replicable way. The authors use one sample of Z during the gradient calculation. Would not it be more consistent then to use the maximum instead of sampling? Or is even getting the maximum is computationally too expensive? Considering the left-to-right approximation, this might be possible and more consistent with your decoding, please address this issue in details.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Latent Sequence Decompositions
Sequence-to-sequence models rely on a fixed decomposition of the target sequences into a sequence of tokens that may be words, word-pieces or characters. The choice of these tokens and the decomposition of the target sequences into a sequence of tokens is often static, and independent of the input, output data domains. This can potentially lead to a sub-optimal choice of token dictionaries, as the decomposition is not informed by the particular problem being solved. In this paper we present Latent Sequence Decompositions (LSD), a framework in which the decomposition of sequences into constituent tokens is learnt during the training of the model. The decomposition depends both on the input sequence and on the output sequence. In LSD, during training, the model samples decompositions incrementally, from left to right by locally sampling between valid extensions. We experiment with the Wall Street Journal speech recognition task. Our LSD model achieves 12.9% WER compared to a character baseline of 14.8% WER. When combined with a convolutional network on the encoder, we achieve a WER of 9.6%.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Interesting paper which proposes jointly learning automatic segmentation of words to sub words and their acoustic models. Although the training handles the word segmentation as hidden variable which depends also on the acoustic representations, during the decoding only maximum approximation is used. The authors present nice improvements over character based results, however they did not compare results with word segmentation which does not assume the dependency on acoustic. Obviously, only text based segmentation would result in two (but simpler) independent tasks. In order to extract such segmentation several publicly open tools are available and should be cited. Some of those tools can also exploit the unigram probabilities of the words to perform their segmentations. It looks that the improvements come from the longer acoustical units - longer acoustical constraints which could lead to less confused search -, pointing towards full word models. In another way, less tokens are more probable due to less multiplication of probabilities. As a thought experiment for an extreme case: if all the possible segmentations would be possible (mixture of all word fragments, characters, and full-words), would the proposed model use word fragments at all? (WSJ is a closed vocabulary task). It would be good to show that the sub word model could outperform even a full-word model (no segmentation). Your model estimates p(z_t|x,z<t;	heta), but during training both p(z_t|x,y,z<t;	heta) and p(z_t|x,z<t;	heta) are needed. Their connections and calculations, in general Section 2 should be more detailed. The experimental setup should be presented in a replicable way. The authors use one sample of Z during the gradient calculation. Would not it be more consistent then to use the maximum instead of sampling? Or is even getting the maximum is computationally too expensive? Considering the left-to-right approximation, this might be possible and more consistent with your decoding, please address this issue in details.
assistant
Interesting paper which proposes jointly learning automatic segmentation of words to sub words and their acoustic models. Although the training handles the word segmentation as hidden variable which depends also on the acoustic representations, during the decoding only maximum approximation is used. The authors present nice improvements over character based results, however they did not compare results with word segmentation which does not assume the dependency on acoustic. Obviously, only text based segmentation would result in two (but simpler) independent tasks. In order to extract such segmentation several publicly open tools are available and should be cited. Some of those tools can also exploit the unigram probabilities of the words to perform their segmentations. It looks that the improvements come from the longer acoustical units - longer acoustical constraints which could lead to less confused search -, pointing towards full word models. In another way, less tokens are more probable due to less multiplication of probabilities. As a thought experiment for an extreme case: if all the possible segmentations would be possible (mixture of all word fragments, characters, and full-words), would the proposed model use word fragments at all? (WSJ is a closed vocabulary task). It would be good to show that the sub word model could outperform even a full-word model (no segmentation). Your model estimates p(z_t|x,z<t;	heta), but during training both p(z_t|x,y,z<t;	heta) and p(z_t|x,z<t;	heta) are needed. Their connections and calculations, in general Section 2 should be more detailed. The experimental setup should be presented in a replicable way. The authors use one sample of Z during the gradient calculation. Would not it be more consistent then to use the maximum instead of sampling? Or is even getting the maximum is computationally too expensive? Considering the left-to-right approximation, this might be possible and more consistent with your decoding, please address this issue in details.