Multi-view Recurrent Neural Acoustic Word Embeddings
Recent work has begun exploring neural acoustic word embeddings–fixed dimensional vector representations of arbitrary-length speech segments corresponding to words. Such embeddings are applicable to speech retrieval and recognition tasks, where reasoning about whole words may make it possible to avoid ambiguous sub-word representations. The main idea is to map acoustic sequences to fixed-dimensional vectors such that examples of the same word are mapped to similar vectors, while different-word examples are mapped to very different vectors. In this work we take a multi-view approach to learning acoustic word embeddings, in which we jointly learn to embed acoustic sequences and their corresponding character sequences. We use deep bidirectional LSTM embedding models and multi-view contrastive losses. We study the effect of different loss variants, including fixed-margin and cost-sensitive losses. Our acoustic word embeddings improve over previous approaches for the task of word discrimination. We also present results on other tasks that are enabled by the multi-view approach, including cross-view word discrimination and word similarity.
This paper proposes an approach to learning word vector representations for character sequences and acoustic spans jointly. The paper is clearly written and both the approach and experiments seem reasonable in terms of execution. The motivation and tasks feel a bit synthetic as it requires acoustics spans for words that have already been segmented from continuous speech - - a major assumption. The evaluation tasks feel a bit synthetic overall and in particular when evaluating character based comparisons it seems there should also be phoneme based comparisons. There*s a lot of discussion of character edit distance relative to acoustic span similarity. It seems very natural to also include phoneme string edit distance in this discussion and experiments. This is especially true of the word similarity test. Rather than only looking at levenshtein edit distance of characters you should evaluate edit distance of the phone strings relative to the acoustic embedding distances. Beyond the evaluation task the paper would be more interesting if you compared character embeddings with phone string embeddings. I believe the last function could remain identical it*s just swapping out characters for phones as the symbol set. finally in this topic the discussion and experiments should look at homophones As if not obvious what the network would learn to handle these. the vocabulary size and training data amount make this really a toy problem. although there are many pairs constructed most of those pairs will be easy distinctions. the experiments and conclusions would be far stronger with a larger vocabulary and word segment data set with subsampling all pairs perhaps biased towards more difficult or similar pairs. it seems this approach is unable to address the task of keyword spotting in longer spoken utterances. If that*s the case please add some discussion as to why you are solving the problem of word embeddings given existing word segmentations. The motivating example of using this approach to retrieve words seems flawed if a recognizer must be used to segment words beforehand
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Multi-view Recurrent Neural Acoustic Word Embeddings
Recent work has begun exploring neural acoustic word embeddings–fixed dimensional vector representations of arbitrary-length speech segments corresponding to words. Such embeddings are applicable to speech retrieval and recognition tasks, where reasoning about whole words may make it possible to avoid ambiguous sub-word representations. The main idea is to map acoustic sequences to fixed-dimensional vectors such that examples of the same word are mapped to similar vectors, while different-word examples are mapped to very different vectors. In this work we take a multi-view approach to learning acoustic word embeddings, in which we jointly learn to embed acoustic sequences and their corresponding character sequences. We use deep bidirectional LSTM embedding models and multi-view contrastive losses. We study the effect of different loss variants, including fixed-margin and cost-sensitive losses. Our acoustic word embeddings improve over previous approaches for the task of word discrimination. We also present results on other tasks that are enabled by the multi-view approach, including cross-view word discrimination and word similarity.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes an approach to learning word vector representations for character sequences and acoustic spans jointly. The paper is clearly written and both the approach and experiments seem reasonable in terms of execution. The motivation and tasks feel a bit synthetic as it requires acoustics spans for words that have already been segmented from continuous speech - - a major assumption. The evaluation tasks feel a bit synthetic overall and in particular when evaluating character based comparisons it seems there should also be phoneme based comparisons. There*s a lot of discussion of character edit distance relative to acoustic span similarity. It seems very natural to also include phoneme string edit distance in this discussion and experiments. This is especially true of the word similarity test. Rather than only looking at levenshtein edit distance of characters you should evaluate edit distance of the phone strings relative to the acoustic embedding distances. Beyond the evaluation task the paper would be more interesting if you compared character embeddings with phone string embeddings. I believe the last function could remain identical it*s just swapping out characters for phones as the symbol set. finally in this topic the discussion and experiments should look at homophones As if not obvious what the network would learn to handle these. the vocabulary size and training data amount make this really a toy problem. although there are many pairs constructed most of those pairs will be easy distinctions. the experiments and conclusions would be far stronger with a larger vocabulary and word segment data set with subsampling all pairs perhaps biased towards more difficult or similar pairs. it seems this approach is unable to address the task of keyword spotting in longer spoken utterances. If that*s the case please add some discussion as to why you are solving the problem of word embeddings given existing word segmentations. The motivating example of using this approach to retrieve words seems flawed if a recognizer must be used to segment words beforehand
assistant
This paper proposes an approach to learning word vector representations for character sequences and acoustic spans jointly. The paper is clearly written and both the approach and experiments seem reasonable in terms of execution. The motivation and tasks feel a bit synthetic as it requires acoustics spans for words that have already been segmented from continuous speech - - a major assumption. The evaluation tasks feel a bit synthetic overall and in particular when evaluating character based comparisons it seems there should also be phoneme based comparisons. There*s a lot of discussion of character edit distance relative to acoustic span similarity. It seems very natural to also include phoneme string edit distance in this discussion and experiments. This is especially true of the word similarity test. Rather than only looking at levenshtein edit distance of characters you should evaluate edit distance of the phone strings relative to the acoustic embedding distances. Beyond the evaluation task the paper would be more interesting if you compared character embeddings with phone string embeddings. I believe the last function could remain identical it*s just swapping out characters for phones as the symbol set. finally in this topic the discussion and experiments should look at homophones As if not obvious what the network would learn to handle these. the vocabulary size and training data amount make this really a toy problem. although there are many pairs constructed most of those pairs will be easy distinctions. the experiments and conclusions would be far stronger with a larger vocabulary and word segment data set with subsampling all pairs perhaps biased towards more difficult or similar pairs. it seems this approach is unable to address the task of keyword spotting in longer spoken utterances. If that*s the case please add some discussion as to why you are solving the problem of word embeddings given existing word segmentations. The motivating example of using this approach to retrieve words seems flawed if a recognizer must be used to segment words beforehand