We give a general framework for inference in spanning tree models. We propose unified algorithms for the important cases of first-order expectations and second-order expectations in edge-factored, non-projective spanning-tree models. Our algorithms exploit a fundamental connection between gradients and expectations, which allows us to derive efficient algorithms. These algorithms are easy to implement, given the prevalence of automatic differentiation software. We motivate the development of our framework with several cautionary tales of previous research, which has developed numerous less-than-optimal algorithms for computing expectations and their gradients. We demonstrate how our framework efficiently computes several quantities with known algorithms, including the  expected attachment score, entropy, and generalized expectation criteria . As a bonus, we give algorithms for quantities that are missing in the literature, including the KL divergence . In all cases, our approach matches the efficiency of existing algorithms and, in several cases, reduces the runtime complexity by a factor (or two) of the sentence length. We validate the implementation of our framework through runtime experiments. We find our algorithms are up to 12 and 26 times faster than previous algorithms for computing the Shannon entropy and the gradient of the generalized expectation objective, respectively .
We propose a general framework for computing expectations in edge-factored, non-projective spanning-tree models. Our algorithms exploit a fundamental connection between gradients and expectations, which allows us to derive efficient algorithms. We motivate the development of our framework with several cautionary tales of previous research, which has developed numerous inefficient algorithms for computing expectations and their gradients. We demonstrate how our framework efficiently computes several quantities with known algorithms, including the Shannon entropy, the expected attachment score, and the generalized expectation criterion . As a bonus, we give algorithms for quantities that are missing in the literature, including the gradient of entropy, the KL divergence, and the gradient of the KL divergence . In all cases, our approach matches the efficiency of existing algorithms and, in several cases, reduces the runtime complexity by a factor of the sentence length. We validate our framework through rigorous proofs of correctness and efficiency .
R
give
propose
clarity
clarity clarity clarity
R
inference in spanning tree models. We propose unified algorithms for the important cases of first-order expectations and second-order expectations
computing expectations
clarity
clarity clarity clarity
D
These algorithms are easy to implement, given the prevalence of automatic differentiation software.
coherence
clarity coherence coherence
R
cautionary tales
cautionary tales
style
style style coherence
R
less-than-optimal
inefficient
clarity
clarity clarity clarity
A
Shannon entropy, the
meaning-changed
meaning-changed meaning-changed meaning-changed
R
entropy, and generalized expectation criteria
and the generalized expectation criterion
clarity
clarity clarity clarity
R
KL divergence
gradient of entropy, the KL divergence, and the gradient of the KL divergence
meaning-changed
meaning-changed meaning-changed meaning-changed
D
(or two)
clarity
clarity clarity clarity
D
the implementation of
clarity
clarity clarity clarity
R
runtime experiments. We find our algorithms are up to 12 and 26 times faster than previous algorithms for computing the Shannon entropy and the gradient of the generalized expectation objective, respectively
rigorous proofs of correctness and efficiency
clarity
clarity clarity clarity