Semantic parsing aims at translating natural language (NL) utterances onto machine-interpretable programs, which can be executed against a real-world environment. The expensive annotation of utterance-program pairs has long been acknowledged as a major bottleneck for the deployment of contemporary neural models to real-life applications. In this work, we focus on the task of semi-supervised learning where a limited amount of annotated data is available together with many unlabeled NL utterances. Based on the observation that programs which correspond to NL utterances must be always executable, we propose to encourage a parser to generate executable programs for unlabeled utterances. Due to the large search space of executable programs, conventional methods that use approximations based on beam-search such as self-training and top-k marginal likelihood training, do not perform as well. Instead, we view the problem of learning from executions from the perspective of posterior regularization and propose a set of new training objectives. Experimental results on OVERNIGHT and GEOQUERY show that our new objectives outperform conventional methods, bridging the gap between semi-supervised and supervised learning. Semantic parsing is the task of mapping natural language (NL) utterances to meaning representations (aka programs) that can be executed against a real-world environment such as a knowledge base or a relational database. While neural sequence-to-sequence models (Dong and Lapata, 2016; Jia and Liang, 2016a) have achieved much success in this task in recent years, they usually require a large amount of labeled data (i.e., utterance-program pairs) for training. However, annotating utterances with programs is expensive as it requires expert knowledge of meaning representations (e.g., lambda calculus, SQLs) and the envi-list all 3 star rated thai restaurants Program Candidates Gold Exe select restaurant where star_rating = thai (cid:55) (cid:55) select restaurant where cuisine > 3 (cid:55) (cid:55) select restaurant where star_rating = 3 (cid:55) (cid:51) select restaurant where star_rating = 3 and cuisine = thai (cid:51) (cid:51) Figure 1: Candidate programs for an utterance can be classified by executability (Exe); note that the gold program is always in the set of executable programs. ronment against which they are executed (e.g., a knowledge base, a relational database). An alternative to annotation is to collect answers (or denotations) of programs, rather than programs themselves (Liang et al., 2013; Berant et al., 2013). In this work, we focus on the more extreme setting where there are no annotations available for a large number of utterances. This setting resembles a common real-life scenario where massive numbers of user utterances can be collected when deploying a semantic parser (Iyer et al., 2017). Effectively utilizing the unlabeled data makes it possible for a semantic parser to improve over time without human involvement. Our key observation is that not all candidate programs for an utterance will be semantically valid. This implies that only some candidate programs can be executed and obtain non-empty execution results. 1 As illustrated in Figure 1, executability is a weak signal that can differentiate between semantically valid and invalid programs. On unlabeled utterances, we can encourage a parser to only focus on executable programs ignoring non-executable ones. Moreover, the executability of a program 1 In the rest of this paper, we extend the meaning of exe-cutability', and use it to refer to the case where a program is executable and obtains non-empty results. can be obtained from an executor for free without requiring human effort. Executability has previously been used to guide the decoding of a semantic parser (Wang et al., 2018). We take a step further to directly use this weak signal for learning from unlabeled utterances. To learn from executability, we resort to marginal likelihood training, i.e., maximizing the marginal likelihood of all executable programs for an unlabeled NL utterance. However, the space of all possible programs is exponentially large, as well as the space of executable ones. Hence, simply marginalizing over all executable programs is intractable. Typical approximations use beam search to retrieve a handful of (seen') programs, which are used to approximate the entire space. Using such approximations can lead to optimization getting trapped in undesirable local minima. For example, we observe that encouraging a model to exploit seen executable programs hinders exploration and reinforces the preference for shorter programs, as discussed in Section 6.3. This happens because shorter programs are both more likely to be among seen' programs (probably due to using locally-normalized autoregressive modeling) and more likely to be executable. To alleviate these issues, we derive three new alternative objectives, relying on a new interpretation of marginal likelihood training from the perspective of posterior regularization. Our proposed objectives encode two kinds of inductive biases: 1) discouraging seen non-executable programs , which plays a similar role to encouraging seen executable ones but does not share its drawback of hindering exploration; 2) encouraging sparsity among executable programs, which encourages a parser to only focus on a subset of executable programs by softly injecting a sparsity constraint. This is desirable, as there are only one or few correct programs for each utterance (see Figure 1), and an accurate parser should assign probability mass only to this subset. We collectively call these objectives X-PR, as a shorthand for Execution-guided Posterior Regularization. We conduct experiments on two representative semantic parsing tasks: text-to-LF (logical form) parsing over a knowledge base and text-to-SQL (Zelle and Mooney, 1996) parsing over a relational database. Concretely, we evaluate our methods on the OVERNIGHT (Wang et al., 2015a) and GEOQUERY datasets. We simulate the semi-supervised learning setting by treating 70% of the training data as unlabeled. Empirical results show that our method can substantially boost the performance of a parser, trained only on labeled data, by utilizing a large amount of unlabeled data. Our contributions are summarized as follows:  We show how to exploit unlabeled utterances by taking advantage of their executability. To better learn from executability, we propose a set of new objectives based on posterior regularization. Our method can help a base parser achieve substantially better performance by utilizing unlabeled data. Our code, datasets, and splits are publicly available at https://github.com/berlino/ tensor2struct-public . Semi-Supervised Semantic Parsing In the context of semantic parsing, semi-supervised models using limited amounts of parallel data and large amounts of unlabeled data treat either utterances or programs as discrete latent variables and induce them in the framework of generative models (Kocisk et al., 2016; Yin et al., 2018). A challenge with these methods is that (combinatorially) complex discrete variables make optimization very hard, even with the help of variational inference. In this work, we seek to directly constrain the discriminative parser with signals obtained from executions. Our method can potentially be integrated into these generative models to regularize discrete variables. (Underspecified) Sequence-Level Rewards There have been attempts in recent years to integrate sequence-level rewards into sequence-to-sequence training as a way of accommodating task-specific objectives. For example, BLEU can be optimized for coherent text generation (Bosse-lut et al., 2018) and machine translation (Wu et al., 2018) via reinforcement learning or beam-search (Wiseman and Rush, 2016). In this work, we resort to marginal likelihood training to exploit binary executability rewards for semantic parsing (i.e., whether a program is executable or not), which has been shown to be more effective than REINFORCE (Guu et al., 2017). More importantly, our binary reward is underspecified, i.e., there exist many spurious programs that enjoy the same reward as the gold program. This issue of learning from underspecified rewards underlies many weakly-supervised tasks, e.g., learning from denotations (Liang et al., 2013; Berant et al., 2013), weakly supervised question answering (Min et al., 2019). Previous work tried to model latent alignments (Wang et al., 2019) between NL and programs to alleviate this issue. In this work, we take an orthogonal direction and propose several training objectives that alleviate the impact of spurious programs. Execution for Semantic Parsing Execution has been utilized in semantic parsing (Wang et al., 2018) and the related area of program synthesis (Chen et al., 2019). These approaches exploit the execution of partial programs to guide the search for plausible complete programs. Although partial execution is feasible for SQL-style programs, it cannot be trivially extended to general meaning representation (e.g., logical forms). In this work, we explore a more general setting where execution can be only obtained from complete programs. In this section, we formally define our semi-supervised learning setting and show how to incorporate executability into the training objective whilst relying on the marginal likelihood training framework. We also present two conventional approaches to optimizing marginal likelihood. Given a set of labeled NL-program pairs { ( x li , y li ) } Ni =1 and a set of unlabeled NL utterances { x j } M j =1 , where N and M denote the sizes of the respective datasets, we would like to learn a neural parser p ( y | x,  ) , parameterized by  , that maps utterances to programs. The objective to minimize consists of two parts: J = 1 NN (cid:88) i =1 L sup ( x li , y li ) +  1 MM (cid:88) j =1 L unsup ( x i ) (1) where L sup and L unsup denote the supervised and unsupervised loss, respectively. For labeled data, we use the negative log-likelihood of gold programs; for unlabeled data, we instead use the log marginal likelihood (MML) of all executable programs . Specifically, they are defined as follows: L sup ( x, y ) =  log p ( y | x,  ) (2) L unsup ( x ) =  log (cid:88) y R ( y ) p ( y | x,  ) (3) where R ( y ) is a binary reward function that returns 1 if y is executable and 0 otherwise. In practice, this function is implemented by running a task-specific executor, e.g., a SQL executor. Another alternative to unsupervised loss is REINFORCE (Sutton et al., 1999), i.e., maximize the expected R ( y ) with respect to p ( y | x,  ) . However, as presented in Guu et al. (2017), this objective usually underperforms MML, which is consistent with our initial experiments. 2 3.2 Self-Training and Top-K MML MML in Equation (3) requires marginalizing over all executable programs which is intractable. Conventionally, we resort to beam search to explore the space of programs and collect executable ones. To illustrate, we can divide the space of programs into four parts based on whether they are executable and observed, as shown in Figure 2a. For example, programs in PSE  PSN are seen in the sense that they are retrieved by beam search. Programs in PSE  PUE are all executable, though only programs in PSE can be directly observed. Two common approximations of Equation (3) are Self-Training (ST) and Top-K MML, and they are defined as follows: LST ( x,  ) =  log p ( y  | x,  ) (4) L top-k ( x,  ) =  log (cid:88) y  PSE p ( y | x,  ) (5) where y  denotes the most probable program, and it is approximated by the most probable one from beam search. It is obvious that both methods only exploit programs in PSE , i.e., executable programs retrieved by beam search. In cases where a parser successfully includes the correct programs in PSE , both approximations should work reasonably well. However, if a parser is uncertain and PSE does not contain the gold program, it would then mistakenly exploit incorrect programs in learning, which is problematic. A naive solution to improve Self-Training or Top-K MML is to explore a larger space, e.g., increase the beam size to retrieve more executable 2 We review the comparison between REINFORCE and MML in the appendix. executable, and N for non-executable. (b) Five objectives to approximate MML. programs. However, this would inevitably increase the computation cost of learning. We also show in the appendix that increasing beam size, after it exceeds a certain threshold, is no longer beneficial for learning. In this work, we instead propose better approximations without increasing beam size. We first present a view of MML in Equation (3) from the perspective of posterior regularization. This new perspective helps us derive three alternative approximations of MML: Repulsion MML, Gentle MML, and Sparse MML. Posterior regularization (PR) allows to inject linear constraints into posterior distributions of generative models, and it can be extended to discriminative models (Ganchev et al., 2010). In our case, we try to constrain the parser p ( y | x,  ) to only assign probability mass to executable programs. Instead of imposing hard constraints, we softly penalize the parser if it is far away from a desired distribution q ( y ) , which is defined as E q [ R ( y )] = 1 . Since R is a binary reward function, q ( y ) is constrained to only place mass on executable programs whose rewards are 1. We denote all such desired distributions as the family Q . the KL-divergence between Q and p , which JQ (  ) = DKL [ Q|| p ( y | x,  )] = min q Q DKL [ q ( y ) || p ( y | x,  )] (6) By definition, the objective has the following upper bound: J (  , q ) = DKL [ q ( y ) || p ( y | x,  )] =  (cid:88) y q ( y ) log p ( y | x,  )  H ( q ) (7) where q  Q , H denotes the entropy. We can use block-coordinate descent, an EM iterative algorithm to optimize it. E : q t +1 = arg min q Q DKL [ q ( y ) || p ( y | x,  t )] M :  t +1 = arg min   (cid:88) y q t +1 ( y )[log p ( y | x,  )] During the E-step, we try to find a distribution q from the constrained set Q that is closest to the current parser p in terms of KL-divergence. We then use q as a soft label' and minimize the cross-entropy between q and p during the M-step. Note that q is a constant vector and has no gradient wrt. during the M-step. where p ( PSE  PUE ) = (cid:80) y (cid:48)  PSE  PUE p ( y (cid:48) | x,  t ) q t +1 ( y ) is essentially a re-normalized version of p over executable programs. Interestingly, if we use the solution in the M-step, the gradient wrt. is equivalent to the gradient of MML in Equation (3). That is, optimizing PR with the EM algorithm is equivalent to optimizing MML. 3 The connection between EM and MML is not new, and it has been well-studied for classification problems (Amini and Gallinari, 2002; Grandvalet and Bengio, 2004). In our problem, we additionally introduce PR to accommodate the executability constraint, and instantiate the general EM algorithm. Although the E-step has a closed-form solution, computing q is still intractable due to the large search space of executable programs. However, this PR view provides new insight on what it means to approximate MML. In essence, conventional methods can be viewed as computing an approximate solution of q . Specifically, Self-Training corresponds to a delta distribution that only focuses on the most probable y  . Top-K MML corresponds to a re-normarlized distribution over PSE . Most importantly, this perspective leads us to deriving three new approximations of MML, which we collectively call X-PR. As mentioned previously, Self-Training and Top-K MML should be reasonable approximations in cases where gold programs are retrieved, i.e., they are in the seen executable subset ( PSE in Figure 2a). However, if a parser is uncertain, i.e., beam search cannot retrieve the gold programs, exclusively exploiting PSE programs is undesirable. Hence, we consider ways of taking unseen executable programs ( PUE in Figure 2a) into account. Since we never directly observe unseen programs ( PUE or PUN ), our heuristics do not discriminate between executable and non-executable programs ( PUE  PUN ). In other words, upweighting PUE programs will inevitably upweight PUN . Based on the intuition that the correct program is included in either seen executable programs ( PSE ) or unseen programs ( PUE and PUN ), we can simply push a parser away from seen non-executable programs ( PSN ). Hence, we call such method Repulsion MML. Specifically, the first heuristic approximates Equation (8) as follows: q t +1 repulsion ( y ) = (cid:40) p ( y | x,  t ) 1  p ( PSN ) y (cid:54) PSN 0 otherwise Another way to view this heuristic is that we distribute the probability mass from seen nonexecutable programs ( PSN ) to other programs. In contrast, the second heuristic is more conserva-tive' about unseen programs as it tends to trust seen executable PSN programs more. Specifically, the second heuristic uses the following approximations to solve the E-step. Intuitively, it shifts the probability mass of seen non-executable programs ( PSN ) directly to seen executable programs ( PSE ). Meanwhile, it neither upweights nor downweights unseen programs. We call this heuristic Gentle MML. Compared with Self-Training and Top-K MML, Repulsion MML and Gentle MML lead to better exploration of the program space, as only seen nonexecutable ( PSN ) programs are discouraged. Sparse MML is based on the intuition that in most cases there is only one or few correct programs among all executable programs. As mentioned in Section 2, spurious programs that are executable, but do not reflect the semantics of an utterance are harmful. One empirical evidence from previous work (Min et al., 2019) is that Self-Training outperforms Top-K MML for weakly-supervised question answering. Hence, exploiting all seen executable programs can be sub-optimal. Following recent work on sparse distributions (Martins and Astudillo, 2016; Niculae et al., 2018), we propose to encourage sparsity of the soft label' q . Encouraging sparsity is also related to the minimum entropy and low-density separation principles which are commonly used in semi-supervised learning (Grandvalet and Bengio, 2004; Chapelle and Zien, 2005). To achieve this, we first interpret the entropy term H in Equation (7) as a regularization of q . It is known that entropy regularization always results in a dense q , i.e., all executable programs are assigned non-zero probability. Inspired by SparseMax (Mar-tins and Astudillo, 2016), we instead use L2 norm for regularization. Specifically, we replace our PR objective in Equation (7) with the following one: J sparse (  , q ) =  (cid:88) y q ( y ) log p ( y | s,  ) + 1 2 || q || 22 where q  Q . Similarly, it can be optimized by the EM algorithm: E : q t +1 = SparseMax Q (log p ( y | x,  t )) M :  t +1 = arg min   (cid:88) y q t +1 ( y )[log p ( y | x,  )] where the top-E-step can be solved by the SparseMax operator, which denotes the Euclidean projection from the vector of logits log p ( y | x,  t ) to the simplex Q . Again, we solve the E-step approximately. One of the approximations is to use top-k SparseMax which constrain the number of non-zeros of q to be less than k . It can be solved by using a top-k operator and followed by SparseMax (Correia et al., 2020). In our case, we use beam search to approximate the top-k operator and the resulting approximation for the E-step is defined as follows: q t +1 sparse = SparseMax y  PSE (cid:0) log p ( y | x,  t ) (cid:1) Intuitively, q t +1 sparse occupies the middle ground between Self-Training (uses y  only) and Top-K MML (uses all PSE programs). With the help of sparsity of q introduced by SparseMax , the M-step will only promote a subset of PSE programs. Summary We propose three new approximations of MML for learning from executions. They are designed to complement Self-Training and Top-K MML via discouraging seen non-executable programs and introducing sparsity. In the following sections, we will empirically show that they are superior to Self-Training and Top-K MML for semi-supervised semantic parsing. The approximations we proposed may also be beneficial for learning from denotations (Liang et al., 2013; Berant et al., 2013) and weakly supervised question answering (Min et al., 2019), but we leave this to future work. In principle, our X-PR framework is model-agnostic, i.e., it can be coupled with any semantic parser for semi-supervised learning. In this work, we use a neural parser that achieves state-of-the-art performance across semantic parsing tasks. Specifically, we use RAT-SQL (Wang et al., 2020) which features a relation-aware encoder and a grammar-based decoder. The parser was originally developed for text-to-SQL parsing, and we adapt it to text-to-LF parsing. In this section, we briefly review the encoder and decoder of this parser. For more details, please refer to Wang et al. (2020). Relation-aware encoding is originally designed to handle schema encoding and schema linking for text-to-SQL parsing. We generalize these two notions for both text-to-LF and text-to-SQL parsing as follows:  enviroment encoding : encoding enviroments, i.e., a knowledge base consisting of a set of triples; a relational database represented by its schema  enviroment linking : linking mentions to intended elements of environments, i.e., mentions of entities and properties of knowledge bases; mentions of tables and columns of relational databases Relation-aware attention is introduced to inject discrete relations between environment items, and between the utterance and environments into the self-attention mechanism of Transformer (Devlin et al., 2019). The details of relation-aware encoding can be found in the appendix. Typical sequence-to-sequence models (Dong and Lapata, 2016; Jia and Liang, 2016a) treat programs as sequences, ignoring their internal structure. As a result, the well-formedness of generated programs cannot be guaranteed. Grammar-based decoders aim to remedy this issue. For text-to-LF parsing, we use the type-constrained decoder proposed by Krishnamurthy et al. (2017); for text-to-SQL parsing, we use an AST (abstract syntax tree) based decoder following Yin and Neubig (2018). Note that grammar-based decoding can only ensure the syntactic correctness of generated programs. Executable programs are additionally semantically correct. For example, all programs in Figure 1 are OVERNIGHTGEO Model BASKETBALLBLOCKSCALENDARHOUSINGPUBLICATIONSRECIPESRESTAURANTSSOCIAL Avg. well-formed, but the first two programs are semantically incorrect. To evaluate X-PR, we present experiments on semi-supervised semantic parsing. We also analyze how the objectives affect the training process. We simulate the setting of semi-supervised learning on standard text-to-LF and text-to-SQL parsing benchmarks. Specifically, we randomly sample 30% of the original training data as the labeled data, and use the rest 70% as the unlabeled data. For text-to-LF parsing, we use the OVERNIGHT dataset (Wang et al., 2015a), which has eight different domains, each with a different size ranging between 801 and 4,419; for text-to-SQL parsing, we use GEOQUERY (Zelle and Mooney, 1996) which contains 880 utterance-SQL pairs. The semi-supervised setting is very challenging as leveraging only 30% of the original training data would result in only around 300 labeled examples in four domains of OVERNIGHT and also in GEOQUERY . baselines, we train two supervised models. The first one only uses the labeled data (30% of the original training data) and discards the unlabeled data in the semi-supervised setting. We view this baseline as a lower bound in the sense that any semi-supervised method is expected to surpass this. The second one has extra access to gold programs for the unlabeled data in the semi-supervised setting, which means it uses the full original training data. We view this baseline as an upper bound for semi-supervised learning; we cannot expect to approach it as the executability signal is much weaker than direct supervision. By comparing the performance of the second baseline (upper bound) with previous methods (Jia and Liang, 2016b; Herzig and Berant, 2017; Su and Yan, 2017), we can verify that our semantic parsers are state-of-the-art. Please refer to the Appendix for detailed comparisons. Our main experiments aim to show how the proposed objectives can mitigate the gap between the lowerand upper-bound baselines by utilizing 70% unlabeled data. Semi-Supervised Training and Tuning We use stochastic gradient descent to optimize Equation (1). At each training step, we sample two batches from the labeled and unlabeled data, respectively. In preliminary experiments, we found that it is crucial to pre-train a parser on supervised data alone; this is not surprising as all of the objectives for learning from execution rely on beam search which would only introduce noise with an untrained parser. That is,  in Equation (1) is set to 0 during initial updates, and is switched to a normal value afterwards. We leave out 100 labeled examples for tuning the hyperparameters. The hyperparameters of the semantic parser are only tuned for the development of the supervised baselines, and are fixed for semi-supervised learning. The only hyperparameter we tune in the semi-supervised setting is the  in Equation (1), which controls how much the unsupervised objective influences learning. After tuning, we use all the labeled examples for supervised training and use the last checkpoints for evaluation on the test set. Our experiments evaluate the objectives presented in Figure 2 under a semi-supervised learning setting. Our results are shown in Table 1. Self-Training and Top-K MML First, Top-K MML, which exploits more executable programs than Self-Training, does not yield better performance in six domains of OVERNIGHT and GEOQUERY . This observation is consistent with Min et al. (2019) where Top-K MML underperforms Self-Training for weakly-supervised question answering. Self-Training outperforms the lower bound in five domains of OVERNIGHT , and on average. In contrast, Top-K MML obtains a similar performance to the lower bound in terms of average accuracy. X-PR Objectives In each domain of OVERNIGHT and GEOQUERY , the objective that achieves the best performance is always within X-PR. In terms of average accuracy in OVERNIGHT , all our objectives perform better than Self-Training and Top-K MML. Among X-PR, Sparse MML performs best in five domains of OVERNIGHT , leading to a margin of 4.2% compared with the lower bound in terms of average accuracy. In GEOQUERY , Sparse MML also obtain best performance. Based on the same intuition of discouraging seen non-executable programs, Repulsion MML achieves a similar average accuracy to Gentle MML in OVERNIGHT . In contrast, Gentle MML tends to perform better in domains whose parsers are weak (such as HOUSING , BLOCKS ) indicated by their lower bounds. In GEOQUERY , Gentle MML performs slightly better than Repulsion MML. Although it does not perform better than Repulsion MML, it retrieves more accurate programs and also generates longer programs (see next section for details). To see how much labeled data would be needed for a supervised model to reach the same accuracy as our semi-supervised models, we conduct experiments using 40% of the original training examples as the labeled data. The supervised model achieves 72.6% on average in OVERNIGHT , implying that labeling' 33.3% more examples would yield the same accuracy as our best-performing objective (Sparse MML). To better understand the effect of different objectives, we conduct analysis on the training process of semi-supervised learning. For the sake of brevity, we focus our analysis on the CALENDAR domain but have drawn similar conclusions for the other domains. Length Ratio During preliminary experiments, we found that all training objectives tend to favor short executable programs for unlabeled utterances. To quantify this, we define the metric of average ratio as follows: ratio = (cid:80) i (cid:80) y  PSE ( x i ) | y | (cid:80) i | x i || PSE ( x i ) | (9) where PSE ( x i ) denotes seen executable programs of x i , | x | , | y | denotes the length of an utterance and a program, respectively, and | PSE ( x i ) | denotes the number of seen executable programs. Intuitively, average ratio reveals the range of programs that an objective is exploiting in terms of length. This metric is computed in an online manner, and x i is a sequence of data fed to the training process. As shown in Figure 3a, Top-K MML favors shorter programs, especially during the initial steps. In contrast, Repulsion MML and Gentle MML prefer longer programs. For reference, we can compute the gold ratio by assuming PSE ( x i ) only contains the gold program. The gold ratio for CALENDAR is 2.01, indicating that all objectives are still preferring programs that are shorter than gold programs. However, by not directly exploiting seen executable programs, Repulsion MML and Gentle MML alleviate this issue compared with Top-K MML. Coverage Next, we analyze how much an objective can help a parser retrieve gold programs for unlabeled data. Since the orignal data contains the gold programs for the unlabeled data, we ultilize them to define the metric of coverage as follows: coverage = (cid:80) i I [ y i  PSE ( x i )] (cid:80) i | x i | (10) where I is an indicator function,  y i denotes the gold program of an utterance x i . Intuitively, this metric measures how often a gold program is captured in PSE . As shown in Figure 3b, Self-Training, which only exploits one program at a time, is relatively weak in terms of retrieving more gold programs. In contrast, Repulsion MML retrieves more gold programs than the others. As mentioned in Section 4.3, SparseMax can be viewed as an interpolation between Self-Training and Top-K MML. This is also reflected in both metrics: Sparse MML always occupies the middle-ground performance between ST and Top-K MML. Interestingly, although Sparse MML is not best in terms of both diagnostic metrics, it still achieves the best accuracy in this domain. In this work, we propose to learn a semi-supervised semantic parser from the weak yet freely available executability signals. Due to the large search space of executable programs, conventional approximations of MML training, i.e, Self-Training and Top-K MML, are often sub-optimal. We propose a set of alternative objectives, namely X-PR, through the lens of posterior regularization. Empirical results on semi-supervised learning show that X-PR can help a parser achieve substantially better performance than conventional methods, further bridging the gap between semi-supervised learning and supervised learning. In the future, we would like to extend X-PR to related tasks such as learning from denotations and weakly supervised question answering. Acknowledgements We would like to thank the anonymous reviewers for their valuable comments. We gratefully acknowledge the support of the European Research Council (Titov: ERC StG Broad-Sem 678254; Lapata: ERC CoG TransModal 681760) and the Dutch National Science Foundation (NWO VIDI 639.022.518).
abstain abstain method objective abstain objective objective abstain abstain other abstain abstain method abstain abstain abstain abstain abstain method objective abstain abstain method abstain abstain abstain abstain abstain result abstain objective objective abstain objective method method method result objective objective result other other other abstain method other other other abstain abstain other other objective other other other objective abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method method method abstain abstain abstain abstain method method method abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method method abstain method method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain method abstain method method other method abstain abstain method abstain method abstain method method abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective abstain objective abstain abstain