Efficient Gradient Computation for Structured Output Learning with Rational and Tropical Losses
Many structured prediction problems admit a natural loss function for evaluation such as the edit-distance or n-gram loss. However, existing learning algorithms are typically designed to optimize alternative objectives such as the cross-entropy. This is because a naïve implementation of the natural loss functions often results in intractable gradient computations. In this paper, we design efficient gradient computation algorithms for two broad families of structured prediction loss functions: rational and tropical losses. These families include as special cases the n-gram loss, the edit-distance loss, and many other loss functions commonly used in natural language processing and computational biology tasks that are based on sequence similarity measures. Our algorithms make use of weighted automata and graph operations over appropriate semirings to design efficient solutions. They facilitate efficient gradient computation and hence enable one to train learning models such as neural networks with complex structured losses.
This paper proposes finite-state approaches for computing a specific loss function (eq 1) and its gradient.  The approach targets improving the training-time efficiency of a finite-state transducer (FST) structured model (e.g., a chain structured CRF) under an rich family FST structured loss function (e.g., edit distance against a gold reference output string). The key contribution of the paper is the efficient computation of the family of structured losses.   Finite-state methods are an obvious choice for performing these computations - it will be familiar/obvious to many NLP researchers (like myself), but perhaps less so in a machine learning venue such as NIPS/ICML. This paper goes beyond the basic construction and includes some useful discussion and correctness details.   Other comments =============  - Why limit to loss functions of the form (1)? It might be useful to consider expected loss, e.g., Li & Eisner (2009) -- adding some discussion of this paper would be a useful pointer for readers.  - If I'm not mistake, the loss function (1) is equivalent to soft-max margin (Gimpel & Smith, 2010; additionally Kevin Gimpel's thesis) and reward-augmented maximum likelihood (RAML; Norouzi et al., 2016). These references contain strong empirical evidence to support the choice of this loss, which are complementary to the theoretical results of reference 7 in the submission.  - Have you considered a REINFORCE-style approximation as another experimental baseline (this is the approach taken in reference 26 in the submission, if I'm not mistaken)? This approach gives an unbiased (but high variance) estimate of a similar (although not the same) objective function (expected reward, which is essentially Li&Eisner'09). If I'm correct that loss function (1) is the same as RAML, then the REINFORCE-style estimator can be modified to compute this loss as well. Another reasonable approximation is to take the K-best outputs as used in Gimpel & Smith (2010), this type of approximation is very common in NLP.  References ========= (Li&Eisner'09) http://www.cs.jhu.edu/~jason/papers/li+eisner.emnlp09.pdf (Gimpel & Smith, 2010) http://www.aclweb.org/anthology/N10-1112 (Norouzi et al 2016) https://papers.nips.cc/paper/6547-reward-augmented-maximum-likelihood-for-neural-structured-prediction.pdf (Kevin Gimpel's thesis) http://ttic.uchicago.edu/~kgimpel/papers/gimpel_thesis.pdf   Post discussion and author response ============================  The authors have addressed my concerns in their response. These concerns were mainly about connections to other work and additional suggestion of experimental comparisons that could be made to strengthen the paper. The empirical evaluation in this paper is kind of silly: they compare an (truly) exponential-time algorithm to their polynomial-time algorithm... no surprise poly-time is much better. I suggested a randomized approximation scheme (based on Monte Carlo sampling / REINFORCE) as a baseline. They didn't bite...  Re: "novelty" of the paper: Most people familiar with finite-state methods (e.g., NLP folks such as myself) would come up with this same approach.  However, these authors did a reasonable good job of "packing up" their method, which will be useful to researcher.  I do wish there was more "tutorial material" to help orient readers that don't know finite-state methods (since this is probably the target audience of the paper). To that end, I **really** like R2's suggestion to include a detailed worked example in the appendix.  I also strongly encourage the authors to release their code in order maximize the impact of their work.  I was too harsh about novelty in my initial assessment so I have increased my overall score to 7.   In regards to some of R1 comments: I wanted to point out that the type of "state-space augmentation" found by finite-state methods is more fine grained than the junction-tree method used on graphical models. There are many cases where the finite-state approach yields exponentially smaller dynamic programs than the best one could do with graphical models. This is discussed in the Sum-Product networks paper (https://arxiv.org/pdf/1202.3732.pdf).  The authors might want to make that connection in their paper.