However, when considering high-capacity neural parametrizations that condition on the whole input sequence, both model classes are theoretically equivalent in terms of the distributions they are capable of representing.

the practical advantage of global normalization in the context of modern neural methods remains unclear.


A typical pattern of pun sentences is that the pun word only occurs once towards the end of the sentence, which separates local context from pun-related topics at the beginning.

we retrieve sentences containing exactly one w a and rank them by the position of w a in the sentence (later is better).


The pun word usually fits in the context as it often has the same part-of-speech tag as the alternative word.

the swap creates local surprisal by putting the pun word in an unusual but acceptable context.


This global association must not be too strong that it eliminates the ambiguity.

we include a single topic word related to the pun word by replacing one word at the beginning of the seed sentence.


For example, changing "person" to "ship" in "Each person must pay their fare share" does not make sense even though "ship" and "fare" are related.

we restrict the deleted word in the seed sentence to nouns and pronouns, as verbs have more constraints on their arguments and replacing them is likely to result in unacceptable sentences.


Creative generation is more challenging as it requires both formality (e.g., grammaticality, rhythm, and rhyme) and novelty.

many works (including us) impose strong constraints on the generative process, such as Petrovic and Matthews (2013); Valitutti et al.


The Road Home, the Louisiana grant program for homeowners who lost their houses to hurricanes Katrina and Rita, is expected to cost far more than the $7.5 billion provided by the federal government, in part because many more families have applied than officials had anticipated.

louisiana officials on Tuesday night set a July 31 deadline for applicants, who can receive up to $150,000 to repair or rebuild their houses.


At the same time, they are noisy and may not reflect the desired phenomenon.

we also conducted an experiment on Amazon Mechanical Turk 6 (AMT) to obtain additional gold references, which are further required by metrics such as BLEU (Papineni et al., 2002).


Human dialogs are like well-structured buildings, with words as the bricks, sentences as the floors, and topic transitions as the stairs connecting the whole building.

discovering dialog structure is crucial for various areas in computational linguistics, such as dialog system building (Young, 2006), discourse analysis (Grosz and Sidner, 1986), and dialog summarization (Murray et al., 2005;Liu et al., 2010).


Moreover, structure annotations on human conversation can be subjective, which makes it hard to reach inter-rater agreements.

we propose an unsupervised method to infer the latent dialog structure since unsupervised methods do not require annotated dialog corpus.


The motivation is that the dialog structure contains important information on the flow of the conversation; if the automatic dialog system can mimic the behaviour in human-human dialogs, it can interact with users in a more natural and user-friendly way.

we propose to integrate the dialog structure information into the reward design of the reinforcement learning (RL).


The ultimate goal of such a structure discovery model is to utilize the structure to facilitate downstream tasks, such as dialog system training.

we propose to incorporate the dialog structure information into the reward function of the RL training.


But this type of delayed reward functions doesn't have immediate rewards at each turn, which makes the model converge slowly.

we propose to incorporate the learned conversational exchange transition information as an immediate reward.


A.2 RL Simulation Setting The main purpose of the user simulator is to provide named entities.

it is built by sampling utterances from the real dataset according to the current status of named entities.


For instance, dependency rela-tions between words in a sentence may be more useful for detecting intra-sentence causal relations, than when used for detecting cross-sentence causal relations.

we train two separate logistic regression classifiers, one for intrasentence causal link detection and the other for cross-sentence causal link detection.


We observed that the cross-sentence causal relation classifier is usually not as capable as the intrasentence classifier, probably due to less contextual evidence to rely on.

for crosssentence event mention pairs that can be converted to intra-sentence cases through event coreference links, we use a heuristic method to improve causal relation prediction performance and replace the predictions from the cross-sentence classifier with the predictions from the intra-sentence classifier 5 , by using system predicted event coreference links.


In addition, the first sentence often summarizes the central context of story and are likely to describe foreground events (Grimes, 1975) that have causal relation with multiple other events.

we add an objective function (equation 7) and additional constraints (equation 8) to encourage causal relations that contain an event from the first sentence.


An event that causes another event must necessarily occur before or temporally overlap with the latter.

clauses having one of these temporal discourse relations may also favor causal relations between events in them.


While this relation between causal and coreference relations is strictly true for gold standard data, we observed that these constraints make the system very sensitive to noise when using system predicted coreference links.

we added binary leakage variables c 1 (i, j, k), c 2 (i, j, k) and c 3 (i, j) to relax these constraints.By maximizing the negative of leakage variables, we allow our model to overcome this instability.


For example, the sentence "it is not a laptop to game with" is more informative than the sentence "I bought this laptop yesterday" in learning the representation of this laptop.

we use sentence-level attention network to help our model select and attend to important sentences to learn more informative review representations.


1, the first review contains much more information of the laptop than the second review, and should has more contributions in building the representation of this laptop.

we use a review-level attention network to distinguish informative reviews from less informative ones.


The user representation learned from the reviews is the summation of the contextual representations of reviews weighted by their attention weights: Although the user representation u r learned from reviews contain rich information of users, there are some latent characteristics of users which are not described in their reviews but can be inferred from the rating patterns.

we also represent users using the embedding of their IDs to capture the latent factors of users, which are motivated by traditional recommendation methods (Koren et al., 2009).


3(b) the sentence "From reading I found short stories are still not my style" is assigned high attention weight since it is informative for representing this user and is important for recommendation, while the sentence "I purchased this book on a whim" has low attention weight since it contains limited information of users and items.

these results validate that our approach is effective in selecting informative words and sentences in reviews for recommendation through the word-and sentence-level attention networks.


Using identity 3 one can easily infer that: The following property needs to be verified: In the proof, we exploit the fact that for every positive-semidefinite matrix X, the following equation holds We observe that for the denominator Putting things together we finally obtain However, proposition 1 is not sufficient in all cases, since negative cosine similarity values can occur in practice.

we also prove a stronger claim stated in the following proposition.


We furthermore have K(E(t) E(u)) ≤ E(t) E(u) for the Frobenius and L 1,1 -norm, since replacing a zero entry with another value can never decrease the value of the norm.

the last inequality follows from proposition 1.


Actually, both are special cases of a norm that raises the absolute values of the matrix components to a certain power e. If this exponent e becomes large, then: (mainly the diagonal elements of the matrices in the denominator assume 1) where #p denotes the number of perfect matches (similarity value of 1.0) between words of the two documents and n (m) is the number of words in text t (u).

with an increasing exponent, sn L e,1 tends to focus on very good matches and disregards the others.


However, such an approach would be infeasible already for medium keyword list sizes.

we use a beam search approach instead, which extends the keyword list iteratively and keeps only the n-best performing permutations.


Actually, 99% of its values fall into the interval [0,1].

the nonboundedness is much less a problem in practice than the theoretical results indicate.


However, we acknowledge the fact that neural models can produce better representations, which can lead to a superior performance.

to collocate our results in a more general setting, we also carried out experiments using the embeddings of questions and passages, produced by an accurate Convolutional Neural Network (CNN) for passage reranking.


Since our baselines, SVM and SVM map , as well as our proposed models, LSP and LSSVM, do not apply such transformations, they may perform lower than the state of the art.

we will use the embedding vectors generated by a CNN to show that they can achieve the state-of-the-art accuracy.


At each of the following steps j,r (7) Note that the loss part, I − N −j l N −j (y,r), in the above formula will be invariant for all the negative items remained in the candidate list, as well as it is for all the positive ones (equal to 0).

,r N −j is essentially the argmin taken over only two candidate items from D \{r N −k } j−1 k=0 : one is the positive item with the minimal weight w • ψ(q, d), and the other, respectively, is the minimal weighted negative one.


1 due to equal max-violated and ground truth rankings) might advance the performance.

good features make our models competitive with the state of the art.


According to our analysis of baseline models, distinguishing between positive and negative examples is the real bottleneck in the task.

we use the direct supervision data to supervise P (e 1 ∼ e 2 | s).


Rest of the layers are processed accordingly.

we can define the above process as follows: from a L layer DC-Bi-LSTM model, the output is The nested LSTMs (NL-STMs) architecture (Moniz and Krueger, 2017) creates a temporal hierarchy of memories that achieved significant improvement over the singlelayer or stacked LSTM architectures to learn longer-term dependencies.


While the value of the outer memory cell in the LSTM is estimated as c outer t = f t c t−1 + i t g t , memory cells of the NLSTMs use the concatenation (f t c t−1 , i t g t ) as input to an inner LSTM (or NLSTM) memory cell, and set c outer t = h inner t .

in compared to the LSTM and stacked LSTM, the inner memories of NLSTMs operate on longer time-scales and effectively capture the context information from the input texts.


In our proposed metric, the word embeddings and the sentence-level tf • idf respectively denote the feature and the weight.

our proposed metric can produce an evaluation based on the word meaning.


All f ij are transported only in the direction from P i to Q j according to Constraint (2).

in each signature this example of Figure 2 conforms to Constraint (5).


However, it is lower than the align score 0.833 between "you" in the translation and "you" in the reference.

the word which corresponds to "that" in the translation cannot be obtained in the reference.


That between "discuss" in the translation and "topics" in the reference is 0.550 ( 8 10 − 3 12 ).

the align score of "discuss" in the translation and "talking" in the reference is 0.445 (0.460 × (1.0 − 0.033)).


The e −pos inf (T i ,R j ) represents the penalty   to cos sim(t i , r j ) because it becomes smaller as pos inf (T i , R j ) becomes larger.

d ij becomes large when the relative difference between the position of word T i in the translation and the position of word R j in the reference (i.e., pos inf (T i , R j )) is large.


However, in the automatic MT evaluation metrics, the score is close to 1.0 when the evaluation for the translation is generally high.

we obtain W E W P I by taking the value of ∑ n j=1 f ij from 1.0.


Therefore, we obtain W E W P I by taking the value of ∑ n j=1 f ij from 1.0.

in between the translation "Are there topics that you think should discuss world?"


The script is available at http://www.statmt.org/wmt16/results.html.


English-to-Turkish (en-tr), and Turkish-to-English (tr-en)).

the WE WPI is effective with such language pairs because it uses word position information.


Note that during both training and testing time, the decoder stops beam search when it satisfies the optimal stopping criteria .

we do not need to penalize the overlength translations since we have already promoted the gold reference to the top of the beam at time step |y| during training.


Third, unlike the majority of prior work (Conneau et al., 2018;Artetxe et al., 2018a;Chen and Cardie, 2018, inter alia), we do not limit our evaluation to the intrinsic BLI task only.

we investigate if embedding reweighting, a recently proposed best practice for BWEs, is useful for extrinsic tasks such as document classification and dependency parsing in multilingual settings.


(2018a) obtain a seed lexicon based on the assumption that for a translation pair w X i , w Z j , the monolingual similarity vectors, √ X i X and Z j Z of translations i, j are (approximately) equal up to a permu-tation.

5 seed translations for a source word i are generated by finding the nearest neighbor based on similarity of monolingual similarities.


First, we propose using n-gram retrieval, with standard Inverse Document Frequency similarity and with dense vector representations, that takes into account local sentence similarities that are critical to translation.

we are able to retrieve useful candidates even for broad domain tasks with little train-test overlap.


Due to the limited amount of training data, models are more likely to suffer from over-fitting.

we use a higher dropout rate of 0.2 and weight decay, another common technique against overfitting, with decay weight set as 10 −3 as the default setting.


The model is trained to map the instructions u to a program z such that executing z in the context c results in the denotation y, which we denote by z c = y.

the program z is a latent variable we must search for at both training and test time.


In this paper, our focus is the search problem that arises at training time when training from denotations, i.e., finding programs that execute to the right denotation.

we would like to focus on scenarios where programs are long, and standard beam search fails.


If the user believes the coordinate prediction based on this advice is wrong, it can tell the model to "retry", and then the second most likely restrictive advice will be used.

the only human feedback needed now is telling the model to "retry", rather than accurate advice as before.


First, they rely on a predefined vector space which is used as a gold standard representation for the entities in a KG.

the quality of these algorithms depends on how well the vector space is represented.


Due to the size of the OpenSubtitles corpus, it was computationally unfeasible to run feature extraction on the entire corpus.

we randomly select 50,000 narrations from the corpus, where each narration consists of between 1 to 50 contiguous lines of dialogue (about the length of a Cookie Theft narration).


training and test data in any language can be prepared simply by lemmatizing a naturally occurring corpus.


3 Strategies to improve NMT with NSD There are many previous studies on using dependency trees to improve NMT (Nguyen Le et al., 2017;Wu et al., 2017).


Our new differentiable sampling algorithm addresses this issue by optimizing the probability that the reference can be aligned with the sampled output, based on a soft alignment predicted by the model itself.

the output distribution at each time step is evaluated with respect to the whole predicted sequence.


While simple and effective, this objective suffers from the exposure bias problem (Ranzato et al., 2015): the model is only exposed to reference target sequences during training, but has to rely on its own predictions at inference.

errors can accumulate along the generated sequence at inference time.


We limited the set of compounds sharing the same head, which we henceforth call head group, to 20 compounds.

we hope to avoid any biases towards particular heads.


The plot suggests that the heads follow a power-law distribution (Zipf, 1965).

one cannot assume that this task could be solved by looking up heads in a finite lexicon with words that often form derogatory compounds in combination with different modifiers.


For deroga-tory compounds, there should be a high frequency of compound mentions but only a low frequency of the constituents co-occurring in a sentence.


First, while emojis are common, the words in their descriptions are more common.

there is more data to pretrain word embeddings than emoji embeddings.


In auxiliary tasks, since the same reviewer can select the same label multiple times, the personalized word embeddings trained through the multitask learning may implicitly include the selection bias of the output labels depending on the reviewers.

to exclude those irrelevant biases from the personalized embeddings, we introduce personalized bias terms to feed-forward layers of each task.


The second one is not surprising since the metadata estimated in the auxiliary tasks are weakly related to each other.

multi-task learning and personalization did not contribute to the improvement of these auxiliary tasks.


This tendency can be explained as follows: In the dataset used in our experiments, words related to five senses such as "mild," "dry" and "soapy" frequently appear and their usages depend on the feelings and experiences of each individual.

these words show high semantic variations.


They argued that, due to the high number of classes, modeling these tasks independently would be inefficient.

they proposed the copula ordinal regression model for multi-task learning and demonstrated that it can outperform various single-task setups.


We can draw an analogy between modeling facial AUs and modeling news media, where each medium expresses a particular bias (political ideology) and can also be associated with a particular level of factuality.

bias and factuality can be analogous to the facial AUs in (Walecki et al., 2016), and represent two aspects of news reporting, each being modeled on a multi-point ordinal scale.


Name  Furthermore, readers can provide their own feedback on existing annotations, and in case of a large discrepancy, annotation is adjusted after a thorough review.

we believe the annotation quality is good enough to experiment with.


We notice there is no standard splitting information provided for both datasets.


However, according to the description of pun in (Miller et al., 2017), each context contains a maximum of one pun.

such a useful structural constraint is not well captured by the simple approach based on the NP tagging scheme.


Constructive reviews, e.g., review #2, often contain in-depth analysis as well as concrete suggestions.

automatically identifying propositions and their types would be useful to understand the composition of peer reviews.


As a result, automatically identifying propositions and their types would be useful to understand the composition of peer reviews.

we propose an argument miningbased approach to understand the content and structure of peer reviews.


1 The same set was used as input to DeConf.

of this process, around 207K additional word senses were introduced in the space for the 155K unique words in WordNet 3.0.


The numerical reasoning NLI challenge dataset is unsplit.

we generate the datasets used for fine-tuning by subsampling 1000 random examples from the entirety of the challenge dataset, and use the remaining examples for evaluation.


In the first layer, we construct k capsules, wherein entries at the same dimension from all feature maps are encapsulated into a corresponding capsule.

each capsule can capture many characteristics among the entries at the corresponding dimension in the embedding triple.


Our CapsE also performs better than ConvKB on these 4 M-M relations.

results  Table 3: Hits@10 on the WN18RR validation set with N = 50 and the initial learning rate at 1e −5 w.r.t each number of iterations in the routing algorithm m and each 10 training epochs.


1, with partial annotations, we already have some knowledge about the unannotated parts.

further annotations of these variables may use the available budget less efficiently; this effect was first discussed in Ning et al.


I k is concave, as reflected by the downward shape of I k I k 1 .

new annotations are less and less efficient for the chain structure, suggesting the usage of ESPA.


However, on one hand, there is still a gap between the I k analysis and the actual system performance; on the other hand, whether the benefit can be realized in practice also depends on how effective the algorithm exploits partial annotations.

it remains to be seen how ESPA works in practice.


Also, the comparison here is in fact unfair to the partial scheme II, because we assume equal cost for both schemes, although it often costs less in a partial scheme as a large problem is decomposed into smaller parts.

the results shown here implies that the information theoretical benefit of partialness can possibly offset its disadvantages for learning.


Since we expect that vector representations generated from posterior hops are more precisely than the ones produced from prior hops, relatively small tolerance to the risk of matching incorrect QA pairs is accepted for posterior hops.

the loss values take increasing weights across hops.


Manual annotation of large corpora from scratch, meanwhile, can be prohibitively expensive.

we propose an active learning solution for named entity recognition, attempting to maximize a custom model's improvement per additional unit of manual annotation.


Scholars in interdisciplinary fields like the Digital Humanities (DH) are increasingly interested in semantic annotation of specialized corpora that invoke many of these challenges.

such corpora cannot easily be annotated automatically with blackbox, off-the-shelf NER models.


The standard, exclusive evaluation framework, by contrast, only measures the accuracy of the final trained model's predictions on a held out test set.

the inclusive framework is relevant to the user who wants to annotate a finite corpus as fast and as accurately as possible by any means necessary, whereas the exclusive framework is relevant to the user who wants to build an NER tool that can generalize well to other corpora.


The contributions of the interface will not be reflected in our human-out-of-the-loop experiments on standard datasets, as these evaluate only the contributions of Ranker and T agger.

reported performances should be considered a lower bound that can often be improved with minimal human intervention.


By not considering uncapitalized OOVs as named entity candidates, it can systematically avoid entire labels which do not take capitalization, such as dates.

while PTDL performs robustly on the GermEval dataset, CAP is only weakly superior to RAND due to the weak correlation between entity status and capitalization.


The proportion of NASH is 40%, while our method is 49%.

it shows our approach is able to produce more efficient hash codes by using Gumbel softmax relaxation because it is very close to the up-bound of information theory for Bernoulli distribution.


However, applying GAN in NLP is challenging due to the discrete nature of the text.

back-propagation would not be feasible for discrete outputs and it is not straightforward to pass the gradients through the discrete output words of the generator.


Furthermore, we have demonstrated that a large hand-crafted grammar can be leveraged to produce large training sets, which improves performance of neural generators substantially.

we argue that the ability to generate high quality text from MRS makes it a good choice of representation for text generation applications that require semantic structure.


The explicit division to sentences in the text plan allows us to realize each sentence plan individually which allows the realizer to follow the plan structure within each (rather short) sentence, reducing the amount of information that the model needs to remember.

we expect a significant reduction in over-and under-generation of facts, which are common when generating longer texts.


We proposed adding an explicit symbolic planning component to a neural data-to-text NLG system, which eases the burden on the neural component concerning text structuring and fact tracking.

while the plan-based system performs on par with a strong end-to-end neural system regarding automatic evaluation metrics and human fluency evaluation, it substantially outperforms the end-to-end system regarding faithfulness to the input.


We did not use any sophisticated methods for ranking tuples in our experiment.

a better ranking algorithm might lead to improved precision.


where the object entity is unknown and it has to be inferred by reading the support documents.

answering a query corresponds to finding the entity a that is the object of a tuple in the KB with subject s and relation r among the provided set of candidate answers C q .


There are two inputs to ASC: an aspect and a review sentence mentioning that aspect.

aSC is close to RRC as the question is just about an aspect and the review is just a review sentence but aSC only needs to output a class of polarity instead of a textual span.


We let the loss on SQuAD be L MRC , which is in a similar setting as the loss L RRC for RRC.

the joint loss of post-training is defined as L = L DK + L MRC .


This deficiency is due to dis-regarding the context of the entities (Singh et al., 2018c,b).

when aiming for annotating entities and relations of short text, it is important to develop an approach which a) is agnostic of the requirement of large training data and b) jointly links entities and relations to its KG equivalence.


In any knowledge graph, a sentence is represented as triple with <subject, predicate, object>.

we rank the candidates by creating a triple consisting of the relation and entity candidates from DBpedia entity candidates and DBpedia relation candidates, then check if these triples exist in the DBpedia KG.


As a result, the distributional representation of words −→ ws i is {x Although the number of assigned words s p i for each character c i is same in one batch, the number varies from batch to batch.

the size of input to the model is not fixed, which is not conducive to batch training.


The diacritization of Arabic is a word-internal property dependent on both character and word-level contexts.

we consider characters as units of representation.


However their results are somewhat complimentary.

voting on the predictions across both systems improved the results further with a 4.49% WER.


Type Definition Encoder Using trainable type embeddings exposes the denoising model to potential data sparsity issues, as some types appear only a few or zero times in the training data.

we also assign each type a vector based on its definition in WordNet (Miller, 1995).


Some previous work has been cautious in pointing out this ill-defined setting (Ravi and Knight, 2009;Christodoulopoulos et al., 2010), and we argue its inappropriateness for scenarios in which the test set is extremely small or even when no annotated reference corpus exists.

the problem of grounding the sequence of states or cluster IDs to POS tags without using any linguistic resource remains unsolved.


However, both criteria prove to be not correlated with tagging accuracy scores.

we propose a cipher model combination strategy in order to leverage the word-order patterns in several PLs, at the cost of an accuracy drop ranging from just 1.15 percentage points to 13.33 points.


The above analysis shows that word order is an important feature to characterize differences between languages.

it should be taken into consideration in the model design.


Similarly, we clip the distance with a maximum threshold of k (which is empirically set to 10), but we do not discriminate positive and negative values.

instead, since we do not want the model to be aware of directional information, we use the absolute values of the position differences: the learnable relative postion representations have k+1 types rather than 2k+1: we have , and .


To be noted, in this Stack-Pointer decoder, RNN is utilized to record the decoding trajectory and also can be sensitive to word order.

we will refer to it as an order-sensitive decoder.


As mentioned in the previous section, the bilingual embeddings for Chinese and Japanese do not align well with English.

we report the results with delexicalizing.


vMF has been used for modeling similarity between two sentences (Guu et al., 2018), which is particularly suited to our purpose here, since we will evaluate our semantic representations in the context of modeling paraphrases (See Sections 4.1 and 4.2 for more details).

we assume q φ (y|x) follows vMF(µ α (x), κ α (x)) and the prior p θ (y) follows the uniform distribution vMF(•, 0).


However, the performances of these works are highly dependent on manually annotated training data while annotation process is time-consuming and expensive.

cross-domain sentiment classification, which aims to transfer knowledge learned on labeled data from related domains * Equal contribution † Corresponding author (called source domain) to a new domain (called target domain), becomes a promising direction.


Based on the cluster assumption in (Chapelle et al., 2009), the optimal predictor is constant on high density regions.

we can utilize two classifiers to provide diverse views to detect points near the decision boundaries and train the generator to create more discriminative features into high-density region.


Second, λ is expected to be negligibly small and can be usually disregarded.

the second term is important quantitatively in computing the target error.


We use such informal crowd-generated posts as text source, in contrast with existing datasets that mostly use formal documents as source such as news articles.

our dataset could less suffer from some biases that key sentences usually locate at the beginning of the text and favorable summary candidates are already inside the text in similar forms.


Moreover, Rotten Tomatoes use multiple movie reviews written by different users as single source text, and one-sentence consensus made by another professional editor as summary.

each pair of this dataset could be less coherent than that of our TIFU, which is written by the same user.


Thus, we regard the body text as source, the title as short summary, and the TL;DR summary as long summary.

we make two sets of datasets: TIFU-short and TIFU-long.


The input of the model is a source text {x i } = x 1 , ..., x N , and the output is a sequence of summary words {y t } = y 1 , ..., y T , each of which is a symbol from the dictionary V. Online posts include lots of morphologically similar words, which should be closely embedded.

we use the fastText (Bojanowski et al., 2016) trained on the Common Crawl corpus, to initialize the word embedding matrix W emb .


As heuristic extractive methods, the Lead-1 uses the first sentence in the text as summary, and the Ext-Oracle takes the sentence with the highest average score of F1 ROUGE-1/2/L with the gold summary in the text.

ext-Oracle can be viewed as an upper-bound for extractive methods.


Second, novel n-gram ratios of Newsroom-Abs and XSum are higher than those of our dataset, even though their data sources are news publications.

we argue that novel n-gram ratios are pretty good but not a sufficient measure to find extractive bias in the summarization dataset.


Due to the presence of a human in the loop, it typically demands real user feedback, but the cost of collecting sufficient data for various settings of our models is prohibitive.

in our experiments, the users were simulated by using the com-  Table 2: ROUGE-1 (R1), ROUGE-2 (R2) and ROUGE-L (RL) achieved by the state-of-the-art models using our sampling strategies evaluated on the Google compression test set.


By the fact that rank(A + B)  rank(A) + rank(B), the ranks of P and Q are upper-bounded by r. As pointed out by Krueger and Memisevic (2017), the parameter matrices of a trained RNN tend to have full rank.

in the experiments, we set r equal to the hidden size d, aiming to allow the adaptive decoder to use full-rank matrices in the recurrent computation.


QGen employs an LSTM (LSTM q ) to generate the token sequence for each question conditioned on h t , which is used to initialise the hidden state of LSTM q .

as input at every time step, QGen receives a dense embedding of the previously generated token w i−1 and the image features I: We optimise QGen by minimising the Negative Log Likelihood (NLL) of the human dialogues and use the adam optimiser (Kingma and Ba, 2015): in our architecture the LSTM q of QGen in combination with the LSTM e of the Encoder form a sequence-to-sequence model (Sutskever et al., 2014), conditioned on the visual and linguistic context -in contrast to the baseline model, where question generation is performed by a single LSTM on its own.


These two tasks are not equally difficult: While the Guesser has to learn the probability distribution of the set of possible objects in the image, QGen needs to fit the distribution of natural language words.

qGen has a harder task to optimize and requires more parameters and training iterations.


For HRE encoder, there are M image vectors as h v candidates, while the number is M × K for the HCIAE encoder; K is the number of "pixels" in the final CNN feature-map.

it is impossible to use all the generated samples in AMFE.


Intuitively, we expect our model to have a lower sample complexity since training our model involves learning fewer parameters.

we hypothesize that our model will generalize better when there is limited amounts of labeled data as compared to larger neural models with a greater number of parameters.


However, we found that the bottleneck for this semi-supervised learning method is the limited variability of given (seen) environments.

to overcome this, we propose to generate novel and diverse environments via a simple but effective 'environmental dropout' method based on view-and viewpointconsistent masking of the visual features.


2018, we take the instruction-aware hidden vectorh t−1 as the hidden input of the decoder instead of h t−1 .

the information about which parts of the instruction have been attended to is accessible to the agent.


The removal of the left chair (marked with a red polygon) from view o t,2 is inconsistent because it also appears in view o t,1 .

the speaker could still refer to it and the agent is aware of the existence of the chair.


6) are rendered from the Matterport3D dataset (Chang et al., 2017), where different colors indicate different types of objects.

dropout on the semantic view feature would remove the object from the view.


This approach incorporates information from both sources, though it does not differentiate them clearly.

the model may struggle to identify which pieces of information are novel with respect to the curated text.


In Mul-tiRC, a question could have more than one correct answer option.

we use a sigmoid function instead of softmax at the final layer ( Figure 1) and regard the task as a binary (i.e., correct or incorrect) classification problem over each (document, question, answer option) instance.


We observe that merely adding more tasks cannot provide much improvement on the target task.

we propose two MTL training algorithms to improve the performance.


Dataset comparisons in Table 1 and performance in Table 2 suggests that NewsQA share more similarity with SQuAD than MARCO.

a MTL system should weight NewsQA samples more than MARCO samples for higher performance.


If one of them is an unknown variable, the problem is solved.

after an OP is applied to the stack specified in (3), the stack state becomes When the stack action selector has decided to push an operand, the operand selector aims at choosing which operand to push.


This model achieved poor performance on our QA tasks (see rows 8a in Table 1 and row 7a in Table 2).

rather than using this NLI model, we trained InferSent on our data by maximizing the inference probability from the input query 5 to the supporting paragraph.


We included this hyperparameter because we observed that InferSent produces a probability distribution over candidate answers where one answer tends to take most of the probability mass, and these scores dominate in the NoisyOr.

the α m weights are set to 1 for  Table 1: Performance on the ARC dataset, measured using precision at 1 (P@1), on both the Easy and Challenge partitions.


Following previous sequence labeling work (Collobert et al., 2011), we introduce a CRF (Lafferty et al., 2001) layer, which has a transition matrix A ∈ R |T |×|T | to give transition scores of adjacent labels.

the score of a label sequence y = y 1:n for a sentence x = x 1:n is calculated as follows: where θ denotes all the parameters and s[y] indicates the dimension of a vector s corresponding to a label y.


For example, assuming that the gold segmentation of the sentence in Figure 1 is "x 1 |x 2 |x 3 x 4 x 5 (= w 1 |w 2 |w 8 )", candidate words for all characters contain their gold words, and those for characters x 1 and x 2 consist only of respective gold words w 1 and w 2 .

the upper and lower bounds for the sentence are 5/5 and 2/5, respectively.


The final embedding of the target word is indirectly affected by the visual information.

the final word embedding can have an advantage of fully utilizing intra-character compositionality from CNN, inter-character compositionality from LSTM, and context information from Skip-gram.


However, this is hard to transfer to languages like English and Vietnamese where single characters are less informative and tokens are much longer, resulting in a larger combinatory label space.

we choose a semi-Markov formalization to directly model segments.


In contrast, we model characterbased POS tagging.

the expected length of our character segments is considerably larger than the expected length of word-based segments for NER.


In a long conversation, there exists the tendency of semantically related words or phrases reoccur across sentences, or there exists topical coherence.

such conversational context information, higher-level knowledge that spans across sentences, provides important information that can improve speech recognition.


While many studies rely only on discrete metadata and/or demographic information, such information is not always available.

it is of interest to learn about the speaker from the language directly, as it relates to the person's interests and speaking style.


First, a speaker's utterances reflect intents, speaking style, etc.

we may build speaker embeddings by analyzing latent modes that characterize utterances in terms of such characteristics, apart from topic-related interests a user might have.


Also, since each device may be used by multiple users, the device address is not a reliable indicator of the user ID.

the ability to profile the user through one conversational interaction is desirable for guiding the socialbot's dialog policy.


The speaker state tracker in our model accumulates the user information as the conversation evolves.

we directly concatenate the speaker state vector h t and the local speaker mode vectorũ t as the user embedding vector at user turn t. Other than using more turns, this is the same DynamicSpeaker-Model configuration as in §3.3.


Our unsupervised dynamic speaker model differs from previous work in that we build speaker embeddings as a weighted combination of latent modes with weights computed based on the utterance.

the model can construct embeddings for any new users and dynamically update the embeddings as the conversation evolves.


Previous work has observed larger gender language differences when the two speakers have the same gender (Boulis and Ostendorf, 2005).

we carry out the group mean tests on the following three sets: 1) all conversations, 2) conversations involving only males or females, and 3) conversations involving both genders.


This is as we would hope: with our fluent model, we want to generate translations that are semantically the same but with disfluencies removed.

similar METEOR scores with similar recall (52) on the fluent references are encouraging.


TPF is 15-dimensional and initialized randomly.

the representation of each word has a dimensionality of 330 in the raw sentence.


Second, sentence-level event factuality is essential for document-level task, which makes sense when document-and sentence-level factuality of events have different values.

we annotate the sentence-level event factuality as follows: CTevents are negated by negative cues.


The PS-and Uu documents only cover 1.39% and 1.20% in our English and Chinese corpus, respectively.

we mainly focus on the performance of CT+, CT-, and PS+.


Compared to Att 2, Att 2+AT considers the adversarial perturbation and training that can alleviate overfitting.

att 2+aT is superior to att 2, which can prove the effectiveness of adversarial training.


On the other hand, most news texts hardly contain bogus and false contents.

in most CT+ documents, sentence-level factuality values are consistent with the document-level value, just as in S10.


We find that in the Chinese corpus these sentences are commonly in the same paragraph and have a strong semantic coherence.

information in adjacent sentences can contribute to the identification of the document-level factuality of the events in the current sentences.


Although document-level event factuality is based on the factuality information in sentences, sentencelevel factuality value of an event only depends on the current sentence, and is likely to have a different value compared to the current document-level factuality.

the joint model can not improve the performance of document-level task.


However, this approach suffered from the issue of noisy labels.

some subsequent studies (Riedel et al., 2010;Hoffmann et al., 2011;Surdeanu et al., 2012) considered distant supervision relation extraction as a multi-instance learning problem, which extracted relation from a bag of sentences instead of a single sentence.


Given a group of bags with the same relation label, we assign higher weights to those bags which are close to other bags in this group.

the representation of bag group g can be formulated as where g k is the k-th row of the matrix G ∈ R h×3dc in Fig.


It should be noticed that a bag group is one training sample in our method.

the model can also be trained in mini-batch mode by packing multiple bag groups into one batch.


The pre-trained model with only intra-bag attentions converged within 300,000 steps in our experiments.

the initial learning rate for training the model with inter-bag attentions was set as 0.001.


Note that the decoding procedures of ATT BL and ATT RA were equivalent when there was only one sentence in a bag.

the improvements from ATT BL to ATT RA can be attributed to that ATT RA calculated intra-bag attention weights in a relationaware way at the training stage.


First, channel-attention extracts richer information from the word embeddings by introducing the channel weights.

it is more suitable when classification tasks become more complicated and a word more likely represents multiple aspects.


In SparseSCVB0, we obtain sparsity by substituting sparse Monte Carlo approximations γ (pseudo) ij for the original SCVB0 variational distributions γ ij .

the justification for this procedure, also used by (Mimno et al., 2012), is that the expected value of an average over one-hot samples from a distribution is equal to that distribution: the overall procedure is still a valid stochastic optimization algorithm.


The retrieved document is assigned with almost the same keyphrases as the upper document.

keyphrases from similar documents (i.e., retrieved keyphrases) can give useful knowledge to guide the generation of keyphrases for the given document.


That means the scorer should determine the relationship between the given document x and the candidate, which is similar to a natural language inference (NLI) problem.

we adapt the most popular NLI model (Parikh et al., 2016) as our scorer.


Based on this assumption, the desirable transition matrix T ∈ R B×B is defined as: m k is a normalization term for class k. Transition matrix H derived from z((m i , c i ); θ z ) should be in keeping with T .

we minimize the cross entropy between T and H: For instance, if T ij is close to 1, H ij needs to be bigger, which results in the growth of A ij and finally optimize θ z (Eq.4).


As the number of parameters in our model is much larger than that of previous studies (Yan et al., 2010(Yan et al., , 2014Wang and Bi, 2017) due to the introduction of both a classifier and a reliability estimator, the model is much harder to train from scratch.

before we start training the model, we first pre-train the classifier using labels predicted by a simpler method as targets, using e.g.


In Table 7, we further consider a scenario with 5 annotators, 4 Although there are 164 annotators in total in this dataset, each instance was labelled by only 10 of these annotators.

we use Krippendorff's alpha which is applicable to incomplete data to measure the inter-annotator agreement.


We also noticed that the objective function would improve when using cross entropy training, and tended to converge faster in our experiments-generally within just a few epochs.

we recommend to use this training method in practice.


If well learned, style relations are useful knowledge for improving the performance, e.g., increasing the probability of Soul Music if we find that it is heavily linked with two high probability labels: R&B and Jazz.

to better exploit style correlations, we propose a novel deep learning approach with two parts: a label-graph based neural network, and a soft training mechanism with correlation based continuous label representation.


The Complex model employs complex number as the KBR embedding.

the score function is defined as f r (h, t) = Re(h T diag(r)t), in which h, r, t are complex vectors andt stands for the conjugate of t. The Analogy model does not restrict the relation matrix to be diagonal.


Based on this strategy, we design two additional constraints on Problem (2).

this method can be viewed as a shrink of the constraint space of the optimization problem.


One reason is that Analogy's constraint is looser than Complex.

it may easily predict wrong facts due to error propagation; 5.


This is because our goal is to predict relation and tail based on the head.

the choice may be harder to make with high tph; 3.


Recently, deep neural networks (DNNs) have been effectively used to learn robust syntactic and semantic representations behind complex structures.

we propose a novel DNN framework which combines Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) and Convolutional Neural Networks (CNN) (LeCun et al., 1989) with a multi-attention layer.


However, it also yields some new errors (about 70 errors).

the difference of F 1 between the baseline model and our RbSP model is only 1.5%, as stated in table 1.


After that, we proceed to compute question-aware memory representationsM k .

we have: Enhancing module We further enhance the question and KB representations by exploiting two-way attention.


Finally, we apply a residual layer (He et al., 2016)  and batch normalization (BN) (Ioffe and Szegedy, 2015), which help the model performance in practice.

we havê 3.4 Answer module Given the representation of question Q which iŝ q and the representation of candidate answers , we compute the matching score The candidate answers are then ranked by their scores.


Since there can be multiple answers to a given question, the candidates whose scores are close to the highest score within a certain margin, θ, are regarded as good answers as well.

we formulate the inference process as follows: where max a ∈A {S(q, M k a )} is the score of the best matched answer andÂ is the predicted answer set.


We find such questions often query for non-factoid information, and that human annotators need to apply a wide range of inferential abilities when answering them.

they can be used to construct highly inferential reading comprehension datasets that have the added benefit of being directly related to the practical end-task of answering user yes/no questions.


The former also considers previously-addressed keys during query updating, but ignores the value representations.

it still requires predefined rules and threshold to artificially add intermediate value representations to update the query.


Intuitively, in KB-QA, masking previously-addressed keys from the query could benefit latter inference, since the model will be able to focus on the next hop, e.g., the movie Taken.

we take into account the query and addressed memories at the t-th hop when updating the query q t+1 for the next hop: where ⊕ denotes the concatenation of vectors.


However, keep in mind that, we do not have gold-standard structured queries for training.

we have to adopt different strategies to find answers in the training and test phases.


Given a hypothesis statement H qa representing a candidate answer, and the set of premise sentences {P i }, our proposed architecture uses the same function f e for two components: (a) a sentence relevance module that scores each P i based on its potential relevance to H qa , with the goal of weeding out distractors; and (b) a relevance-weighted aggregator that combines entailment information from multiple P i .

we build effective entailment aware representations of larger contexts (i.e., multiple sentences) from those of small contexts (i.e., individual sentences).


Similarly, Concat would also be muddled by distracting information in P2, which will weaken its confidence in answer Cambridge.

without careful guidance, simple aggregation can easily add distracting information into the premise representation, causing entailment to fail.


OpenBookQA, on the other hand, has multiple-choice science questions with exactly one correct answer choice and no associated paragraph.

this dataset requires the relevant facts to be retrieved from auxiliary resources including the open book of facts released with the paper and other sources such as WordNet (Miller, 1995) and ConceptNet (Speer and Havasi, 2012).


On the other hand, if the relevant sentences are far apart, the model finds it difficult to handle such long-range cross sentence dependencies in its contextualized representations.

it ends up focusing attention on the most relevant sentence, missing out on other relevant sentences (Figure 4b).


The important question remains as to how polarized ideologies surrounding race take shape on forums such as Twitter.

in all of the analyses throughout this paper we consider the race of the shooter as a potential factor.


Since we only assign one topic to each tweet, we would like to evaluate how coherent the tweets are that are assigned to the same topic.

we define the tweet intrusion task analogously to word intrusion.


However, such kind of prior knowledge is not available in our decipherment task.

our task is more challenging and is not simply a definition modeling task.


The posterior network and the prior network have the same structure and both output a probability distribution of latent variable z.

the only difference is that the input of the posterior network is the concatenation of the encoded definition x and the condition vector c while the input of the prior network is only the condition vector c. the output of the posterior network p = p α (z|d * , u, s) and the output of the prior network p = p β (z|u, s).


The decoder in our Variational Decipher model is to model the likelihood , which is conditioned on the latent variable, tweet context, and the symbol.

for the Variational Decipher, the condition vector c and the sampled latent variable z are fed into the decoder.


We assume that such types of hate symbols share similar surface forms or similar tweet contexts.

the first testing dataset D s is to evaluate how well the model captures the semantic similarities among the tweet contexts in different examples or the similarities among different surface forms of a hate symbol.


Nearly 70% of the relations are longtail in the widely used New York Times (NYT) dataset 1 (Riedel et al., 2010;Lei et al., 2018) as shown in Figure 1.

it is crucial for mod-els to be able to extract relations with limited numbers of training instances.


Dealing with long tails is very difficult as few training examples are available.

it is natural to transfer knowledge from data-rich and semantically similar head classes to data-poor tail classes .


For instance, the relation /people/person/place lived and /people/person/nationality are more relevant, whereas the relation /people/person/profession has less correlation with the former two relations.

it [ismail_merchant], whose filmmaking collaboration with james ivory created a genre of films with visually sumptuous settings that told literate tales of individuals trying to adapt to shifting societal values , died yesterday in a [London] hospital [darren_mcgavin] , an actor with hundreds of television , movie and theatrical credits to his name , died on saturday in [los_angeles] .


The latter way is to train relation extraction models with abundant unlabeled instances but restricted by the prerequisite of an accurate dataset used as ground truth.

we propose a novel semi-distant supervised approach by integrating both ways to decrease the influence of false negative instances for better relation extraction.


The convolutional kernels are weight vectors represented by W ∈ R d×k(w+p) and the number of kernels is d. In summary, the convolu-tional operation follows the equation,  (Zeng et al., 2015) improves the max-pooling operation with a piecewise method whose outputs of convolutional filters are divided into three segments by head and tail entities.

the max pooling procedure is performed in three segments separately.


With the sentence encoder, we can control relation features contained in the generated representations.

the discriminative module D will try to distinguish labeled data from generative data, while the generative module G makes p(x gen ) ≈ p(x l ).


Coreference resolution groups spans referring to the same entity into one cluster.

we might expect that knowledge learned from one task might benefit another.


Our key intuition to solve this problem is that different predicates often share some visual features, such as being aligned vertically or horizontally, sharing the same font, size or color, and so on.

if we can identify at least one (predicate, object) pair on the page, we can look for other similarly formatted pairs of nodes and assume that they also represent (predicate, object) pairs.


Simply combining in-domain training data with out-of-domain data will lead to overfitting to the out-of-domain data.

some domain adaptation technique should be adopted to improve in-domain translation.


During experiments, we found that the shared encoder-decoder played a major role in the model and the size of the private encoder-decoder didn't influence the results too much.

we just set the size of the private encoder-decoder one-quarter of the shared encoder-decoder considering the training and decoding speed.


In discrete spaces such as natural language sentences, the situation is more problematic; even a flip of a single word or character is generally perceptible by a human reader.

most of the mathematical framework in previous work is not directly applicable to discrete text data.


For a breakdown of the correlation coefficients according to number of perturbation and type of constraints, we refer to Appendix A.3.

in the following, we report attack results both in terms of chrF in the source (s src ) and relative decrease in chrF (RDchrF) in the target (d tgt ).


M X is used to construct a small subset of the parameters in the softmax layer: and W p ∈ R K×d and b p ∈ R K are used instead of W p and b p in Equation 4and 5.

in mini-batched processes with a mini-batch size B, our method constructs B different sets of (W p , b p ).


Although domain experts can achieve a high performance in these challenging tasks, it is not always feasible to rely on limited and expensive human input for large-scale data sets.

if we have a model with 70% prediction accuracy, it is intuitive to ask what percentage of the data should be handed to domain experts to achieve an overall accuracy rate above 90%?


(2011) notes that on sentence compression, longer sentences are perceived by human annotators to preserve more meaning than shorter sentences, controlling for quality.

the drop in human-judged adequacy may be related to our sentences' relatively short lengths.


In other words, if for each document, RNN finds one good way of ordering relevant labels (such as hierarchically) and allocates most of the probability mass to the sequence in that order, the model still assigns low probabilities to the ground truth label sets and will be penalized heavily.

the model has little freedom in discovering and concentrating on some natural label order.


Training an RNN model with the proposed objective (5) requires summing up sequence (permutation) probabilities for a set y, where |y| is the cardinality of the set.

evaluating this objective exactly can be intractable.


However, the fact is obviously true in the KBs.

hRERE-full gives the underlying relation according to the KBs.


Although sisg is powerful, it requires the information of word boundaries as its input, that is, semantic units need to be specified when encoding targets.

it cannot be directly applied to unsegmented languages.


The relation NA is very ambiguous since instances have no unified pattern.

we cannot decide whether a sentence belongs to NA only by the fact that it does not express any other positive relation.


Action We desire the agent to distinguish whether the current sentence is mislabeled or not.

the action of our agent is defined as a i ∈ {0, 1}, where 0 indicates the sentence is incorrectly labeled and 1 indicates the sentence is correctly labeled.


As previously mentioned, the actions of our agent redistribute noisy data into labeled data and unlabeled data, corresponding to correctly labeled and incorrectly labeled instances.

the average likelihood of labeled data will be larger than unlabeled data when the agent makes correct actions.


We define the difference of likelihood between them as the reward to evaluate the performance of our policy.

the reward is defined as follows: where L and U is the subset of labeled data and unlabeled data respectively, and y is the relation label given by DS.


Additionally, the entire DS dataset D is split into labeled data D l and unlabeled data D u .

we utilize the above data to train SemiVAE model and obtain a robust relation classifier which explicitly learns from correctly labeled data and correct incorrectly labeled data implicitly.


G banana is the set defined in the example above plus the word banana itself).

given g w i , FastText scoring function for a word w i and a context w c is defined as follows: the representation of w i is expressed through the sum of the representations of each of the n-grams derived from w i plus the representation of w i itself.


The scoring functionŝ is defined as follows: whereĜ wm := G wm \ {w m }.

the scoring function is defined as the dot product between the sum of input vectors of the subwords of w m and the input vector of w e .


In fact, it is the only component of the loss function which attempts to learn these relationships.

decreasing it's importance (by increasing the value of α) is reflected by a decay in the semantic analogies score.


We build a training set by pairing the mentions sets of the entity-entity pairs from a KB, following the idea discussed in the next section.

we can reduce the analogy task to a binary classification problem, so that p = P ((A, B), (C, D); Θ) is equal to 1 if A : B = C : D, 0 otherwise.


For each dataset, we select the 20% of less frequent relations sorted by the number of entity pairs, having at least 20 instances.

we collect three different one-shot test sets of 92, 55 and 11 unseen relation types for T-REX, CC-DBP and NYT-FB, respectively.


The Wikipedia abstracts consist of well-written contents, typically the definition of one of the two entities in the pairs.

t-REX can be considered an easier dataset compared with the other two.


This also means that, without regularization, the noise model has less ability to diffuse the diagonal elements which leads to poor classification performance.

we use a regularizer (l 2 ) to diffuse the diagonal entries.


We approximate that it takes about one minute to obtain n-gram statistics from the Wikitext-103 training corpus, for example.

we set up our experiments to equalize neural network training time, and we ignore the small onetime cost of computing n-gram statistics.


If we look at results comparing original DVAE prediction based on the encoder and the decoder, both result in similar F1 and NMI numbers.

we can only conclude that currently in the way we do sampling, constraining over encoder is a better choice.


Although we assume that there is no overlapped relation between the KB and the training text corpus, in practice, we may find a lot of applications that the relations are partially observed in KB.

we also test a setting when the KB has different proportions of overlapped relations with training text corpus.


Our initial experiments with iterative decoding showed that increasing beam sizes beyond 4 did not yield improvements in performance.

we report all results using a beam size of 4.


This leads to several difficulties including 1) How to lead the target agent to a bad state and 2) how to force the target agent to make a wrong decision.

previous methods for attacking static models cannot be directly applied.


However, in our work, one agent's input depends on the other agent's output, which makes the input undecidable in the beginning.

an adversarial sentence or example is not enough to conduct attack in dialogue sys-tems.


It is important to keep the agent producing sentences that are correct both grammatically and semantically and keeping them competitive at the same time.

a common strategy is to train agents using supervised learning to learn natural language and to use reinforcement learning to optimize models' performance using on goaloriented learning.


The second model is called the reinforcement learning model (RL agent).

when doing selfplay between RL agent and SV agent, we could get RL agent with 5.86 perplexity, 89.57% agreement and 7.23 average score, while SV agent achieves 5.47 perplexity and 4.55 average score.


In other words, we assume that e s e s and e r e r .

to calculate the effect of the attack, ψ(s, r, o) − ψ(s, r, o), we need to compute e o − e o , followed by: where z s,r = f (e s , e r ).


Our proposed method is similar to KV-MemNN, but with a different definition of key-value and different information distilling process.

we propose a carefully designed inter-sentence attention mechanism to handle it.


Characters "欧" and "洲" also have strong connections, as seen in Figure 3a, because "欧洲" represents "Europe" in Chinese.

both experiment results and visualization verifies that the Convolutional Attention is effective in obtaining phraselevel information between adjacent characters.


When scoring the CO-ORD, we take the left conjunct and the right conjunct which are linked by the CC.

in the rule (2), the conjunct pair linked by a CC-SUB is the incoming CONJ and the leftmost CONJ in the child COORD.


To train the set of parameter θ of our neural networks, we minimize the following loss function: where D is a set of pairs of a sentence and its correct coordinate structures in a training dataset.


Their model decides the boundary of a coordinate structure individually, which may cause conflicts with that of other coordinate structure(s).

we extend their model to find the best combination of coordinate structures, greedily choosing most probable boundaries without conflicts 4 .


To identify the semantic units of a sentence and to correctly understand syntactic relations, it is not sufficient to target only PASA.

we propose a multi-task learning model that effectively leverages ENASA and improves PASA.


Moreover, the event-noun also has a nominative argument role for the predicate mijikai "short".

given this, we prepare a task-specific word embedding matrix that addresses the task-specific distribution of words.


These studies showed that performance improves when low-layer RNN representations are trained in easy tasks and high-layer RNN are leveraged for difficult tasks.

we construct a network that hierarchically overlaps a task-specific RNN on a task-independent RNN.


Since this does not work optimally, especially for smaller data sets, we have decided to give the neural network this type of information as well.

we combine the advantages of classical feature engineering and neural networks.


Second, fake news is typically produced by connecting disjoint pieces of news and unlike well-established journalism (e.g., New York Times) fake news production lacks a meticulous editorial board.

by incorporating the hierarchical discourse-level structure, we can investigate the coherence of fake/real news documents (we will show this later).


The maximum value is this block is 0.4 and corresponds to entry (s 1 ,s 2 ).

the Algorithm 1: The proposed algorithm for discourse dependency tree construction Input: A; r Output: Discourse dependency tree sentence s 2 is added as the child node of the sentence s 1 .


According to the description of the dependency tree in Section 3.1, leaf nodes are isolated discourse units and no other discourse units depend on them.

the more the number of leaf nodes is, the less inter-linked the discourse units will be, and vice versa.


Thus, the more the number of leaf nodes is, the less inter-linked the discourse units will be, and vice versa.

property 1 is likely to indicate the coherence of a documentthe higher p l , the more isolated sentences and the less coherent the document.


The preorder traversal of the tree in Figure 1 is the sequence {s 1 , s 2 , s 4 , s 3 , s 5 } and the sentential sequential order is {s 1 , s 2 , s 3 , s 4 , s 5 }.

according to the definition of Property 2: The preorder traversal of a document's discourse dependency tree takes into consideration the organization of a document respect to the dependencies between sentences.


For instance, in our running example, the parent node s 3 has p position = 3 (i.e., it is the third sentence) and its child node s 5 has c position = 5.

their parent-child distance is |5 − 3| = 2.


Intuitively speaking, usually, we expect that a child node and its parent to be close to each other in the original sequential order.

the less value of this property is, the more coherent a document is likely to be.


As described in Section 4, all three properties are closely connected to the coherence of a document.

real news documents indicate more degree of coherence.


(2018a) showed that responses ranked higher by humans are more similar to the prompt sentence vector.

we compute the cosine similarity between input and response representations.


Prior work has shown that commonly used metrics for overall quality in other generation tasks such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), ME-TEOR (Banerjee and Lavie, 2005) and perplexity have poor correlations with human judgment (Liu et al., 2016;Tao et al., 2018) 4 or are modeldependent (Liu et al., 2016).

we adopt several metrics that evaluate multiple aspects of responses, and also conduct human evaluation for each result we present.


Our goal is to evaluate the effect of using our various training datasets on sentence encoding, given encoders of equivalent capacity and similar setups.

we follow the exact setup of Infersent (Conneau et al., 2017), also used in the Dissent (Malmi et al., 2018) model: we learn to encode sentences into h with a bi-directional LSTM sentence encoder using element-wise max pooling over time.


However, we observed that such a simple approach could easily lead to under-sized vocabulary or oversized vocabulary issues.

we are interested in understanding how the end-task classification accuracy is related to the vocabulary size and what is the minimum required vocabulary size to achieve a specific performance.


However, the Bernouli distribution is hard to reparameterize, where we need to enumerate 2 V different values to compute the expectation over the stochastic dropout vector b.

we follow Wang and Manning (2013) to use a continuous Gaussian approximation, where the Bernouli noise b is replaced by a Gaussian noise z: where It is worth noting that α and p are one-to-one corresponded, and α is a monotonously increasing function of p. For more details, please refer to Wang and Manning (2013).


2015;  to re-interpret the input noise as the intrinsic stochasticity in the embedding weights B itself as follows: where , where the random weights in each row has a tied variance/mean ratio α i .

we rewrite the evidence lower bound as follows: where P(B) is the prior distribution and N (µ, σ) denotes the Gaussian approximate posterior with parameters µ and σ. L(B, θ) is used as the relaxed evidence lower bound of marginal log likelihood log L(f θ (x), y)).


We propose to select vocabulary subset based on the dropout ratio by using a threshold α T .

the remaining vocabulary subset is described as follows: where we useV to denote the subset vocabulary of interest, by adjusting α T we are able to control the selected vocabulary size.


Evaluation Speed As we know, at each vocabulary point, the network needs to perform once evaluation on the whole test set.

it is not practical to draw each vocabulary size from 1 to V and perform V times of evaluation.


Uniformly sampling the data points are proved wasteful, since when the accuracy curve will converge to a point very early, most of the sampled point is actually getting equivalent accuracy.

we propose to increase the interval exponentially to cover more samples at extremely low vocabulary size.


However, they did not explicitly prove the effectiveness of OOV word embeddings.

it is still unclear how well the combination of character N -grams can reconstruct appropriate embeddings for OOV words.


Then, we assume that the following relation always holds between e w and E: e w = E[z e ] where z e = ζ(w).

( 2)the i-th column vector in the matrix E represents the word embedding of the corresponding word w that satisfies i = ζ(w).


However, smaller subword settings, such as row (b), might markedly degrade the performance from the original word embeddings.

it is necessary to discover a better balance between the memory requirement (or the total number of embedding vectors) and performance.


Here the prior distribution ( ) is a standard Gaussian  (0, ), and, by assuming i.i.d training datapoints, the likelihood term is (| ) = ∏ ∈ ( | ) where = ( , ) is a training instance.

the above integral is intractable, however an empirical estimate can be computed using  samples from the posterior, efficient generation of posterior samples is the key for applying the Bayesian treatment.


One simple approach to inject non-stationarity is to discount the relevance of previous observations.

we introduce a tunable decay ratio γ, and modify Eq.


2 Manual tuning of this mixing ratio via a large grid search over the hyperparameter values is very time and compute expensive (even when the number of selected auxiliary tasks is small, e.g., 2 or 3).

in our second stage, we instead apply a nonparametric Bayesian approach to search for the approximately-optimal mixing ratio.


The reward R(Ŷ s ) is then defined as the difference between the sequence-level accuracy (A) of the sampled sequenceŶ s and the baseline sequenceŶ b .

the corresponding self-critical policy gradient loss is: minimizing the RL loss is equivalent to maximizing the likelihood of the sampledŶ s if it obtains a higher sequence-level accuracy than the baselineŶ b .


More interesting intent categories are a direct use of a method or comparison of results.

our dataset provides a concise annotation scheme that is useful for navigating research topics and machine reading of scientific papers.


We observe that the expected number of times (x, y, c) is used as a corrupt x example is k x P (x, •, •)#(•, y, c), since (x, y, c) can only be created as a corrupt x example by randomly sampling x from an example that already contained y and c. The number of times (x, y, c) is used as a corrupt y or c example can be derived analogously.

the global objective of our trenary negative sampling approach is:  With the global objective, we can now ask what is the optimal value of S x,y,c (20) by comparing the partial derivative of (17) to zero.


• Social Identity refers to people's selfconcept of their membership in a social group, and people have an affinity for their groups over others, similar to name mentions in Rosenthal and McKeown (2017).

if a loan request comes from their own groups, they are more likely to contribute, such as "For those of you in our team who love bread, here is a loan about bakery."


While this strategy worked for this example, it could become a problem when parsing files that contain year-less formatted dates.

future improvements will include a numerical disambiguation module to aid in determining if a numerical phrase is temporal.


may not be considered a valid representation of an AMPM entity.

chrono specifically includes the period in the span only if there is a period after each letter in strings (e.g.


Some mentions of "date" were valid temporal expressions, but there were few of them.

we decided to continue to exclude this token.


Additionally, different medical forms, such as pathology reports versus clinical notes, have specific ways to convey dates.

the range of temporal expressions Chrono now identifies has been significantly expanded due to the variety incorporated in the clinical texts.


While the concept of time is the same regardless of the domain, its representation can vary.

temporal parsing provides a good backdrop for determining the challenges of cross-domain application, which is difficult for many NLP applications.


For instance, postprandial plasma glucose was another outcome reported in the aforementioned trial report, and placebo plus glimepeiride was considered as another comparator.

we instructed prompt generators to create multiple prompts for each full-text article.


In either case, which spans are relevant will depend on the prompt assessed.

we consider and contrast variants that condition evidence span prediction on the input prompt.


Consequently, due to the noise in label dependencies, and hence, in the inherent conversational structure, the model is not able to yield as big of a gain on the MRDA as it does on the SwDA.

learning long-range dependencies is a challenge because of noisier and longer path lengths in the network.


In this paper, we present our novel multi-level memory architecture that overcomes the limitations of existing methods: (i) We separate the memory used to store tokens from the input context and the results from the knowledge base.

we learn different memory readers for context words as well for knowledge base entities (ii) Instead of using a subj − rel − obj store, we develop a novel multi-level memory architecture which encodes the natural hierarchy exhibited in knowledge base results by storing queries and their corresponding results and values at each level.


We propose the use of separate memories for copying values from context and KB results.

the model learns separate memory readers for each type of data.


A user's preferences may change as the dialogue proceeds, and can also refer to previously presented queries (non-sequential dialog).

to store multiple queries, we require 3 levels in our multi-level memory as compared to 2 levels in the other datasets, since they don't have more than one query.


This information is already accessible in the hidden state of the LSTM.

mentions cluster into entities already at the hidden layer h i , with no real difference with the query layer q i (see Figure 7).


As we use the cosine normalization, all prediction scores are mapped into the range [-1, 1].

it's not proper to use log-Sigmoid loss function as in SHORTLISTER.


Generally, a good dialogue can be described as an exchange of information that sustain coherence through a train of thoughts and a flow of topics.

a plausible way to evaluate open-ended dialogue systems is to measure the consistency of the responses.


The essence of neural response generation models is designed by maximizing the likelihood of the target response given source utterances.

a dialogue generation task can be formulated as a next utterance prediction problem.


Without constraints, the sequence encoder can only extract sequential features for both t and s based on their final training signals while these features are not necessarily beneficial to the target language.

we further design sentence level adversarial transfer to encourage the encoder to extract language-agnostic sequential features.


Regarding all n-grams of tokens in the monolingual data as phrases and searching the translations for each phrase can be extremely costly.

we extract meaningful source and target phrases from their respective monolingual data, following the work by Marie and Fujita (2018).


NMT architectures expect parallel sentence pairs as training data, even if we have accurate phrase pairs, they cannot be used directly for training.

we propose an algorithm for extracting sentence pairs from the monolingual data that matches the best possible combinations of phrase pairs from the induced phrase table.


It tends to favor very short sentences that are potentially less exploitable for NMT systems, or not even be sentences as it may happen when dealing with noisy monolingual data.

4 we use a slightly modified version which does not penalize uncovered words on the target side in order to favor the extraction of longer target sentences that may contain more translated tokens.


However, it would introduce a strong bias that encourages the retrieval of partial translations similar to the parallel data that are already used to train the system.

the extracted partial translations may be less useful than those extracted with an induced phrase table.


In fact, there exist infinite number of possible mappings that can match the same target distribution.

to learn better mappings, we need to enforce more constraints to our objective.


The results support our hypothesis that word-order divergence can limit the benefits of multilingual translation.

reducing the word order divergence improves translation in extremely low-resource scenarios.


Also, Vietnamese belongs to a different language family than English.

information relevant to the synonymy-antonymy distinction may be distributed differently in the target language space than in the source language.


'bank', 'coast') and phrasal verbs ('stand up', 'stand out') are likely to behave less consistently in multilingual vector spaces due to their different usage distributions.

using such words in the alignment dictionary may result in suboptimal overall mapping.


Intuitively, a sentence is less ambiguous than stand-alone words since the words are interpreted within a specific context, so a mapping learned at the sentence-level is likely to be less sensitive to individual word inconsistencies.

we learn the mapping as described in Section 3.1 using a dictionary of aligned sentence embeddings.


The GTP and FAS models perform better on rules which switch words within the gold-standard summary.

the decoder LSTM has captured the data distribution very well for words within the summary but is not generalizing for words outside it.


1 The trend has made many publishers and editors shift their focus to the web and experiment with new techniques to lure an Internet-savvy generation of readers to read their news stories.

there has been a noticeable increase in the sharing of short illustrative pieces of texts about the news on social media.


A comparison of different seq2seq methods for teaser generation shows a seq2seq combining two levels of vocabularies, source and corpus, is better than one using only the corpus level.

we set a strong baseline on the teaser generation task with a seq2seq model of See et al.


Also, the mean and standard deviation of teaser-article overlap ratio is nearly equal in all the buckets, i.e., 0.559±0.148, 0.559±0.146, 0.564±0.146, 0.566±0.142, 0.566±0.146, respectively.

the range of buckets corresponds to a range in the percentage of uncommon words.


We took a closer look at some of the generated examples, see Table 12, and observed frequent cases where the generation suffered from the typical seq2seq issues, e.g., repetition of words; however, there are also cases where generation is more distinctive than ground-truth and is well formed too.

we carried out a small user study to understand the quality of the generated teasers; however, we only selected non-repeating and non- • pres .


An overlap-based comparative study of headlines and teasers shows teasers as abstractive while headlines as nearly extractive.

we performed neural abstractive summarization studies on teasers and set a strong benchmark on the novel task of teaser generation.


However, we successfully train anchor-based topic models with thousands of topics.

for our fine-grained models, we will employ the Anchor Word algorithm (Arora et al., 2013).


Unlike the separate approach, this method can be considered multi-task learning of the summary and the chit-chat part generation.

we expect it can generate the initial utterance precisely by considering the coherence between the summary and the chit-chat part.


Input news posts that appear in the training data were removed from the test data.

4,776 data were used as the final test data.


Although Baseline (Gen+MMI) achieved high ROUGE scores in Table 4, its style is not colloquial.

workers felt odd and lowered their scores.


Since the embeddings for different lengths are independent, it is hard to generate a sequence of the length that is infrequent in training data.

a method that can model any lengths continuously is required.


In contrast, we have no English test sets that contain headlines of multiple lengths.

we randomly extracted 3,000 sentence-headline pairs that satisfy a length constraint from the test set constructed from annotated English Gigaword (Napoles et al., 2012) by pre-processing scripts of Rush et al.


Since the proposed method consists of a character-based decoder, it sometimes generated  words unrelated to a source sentence.

we applied a simple re-ranking to each n-best headlines generated by the proposed method (n = 20 in this experiment) based on the contained words.


Besides, considering the chain rule in the backward propagation, the complexity of computing gradients for a hierarchical recurrent model increases exponentially from the top layer of the model down to the bottom layer.

to improve the training efficiency of hierarchical recurrent models, our strategy is to simplify the baseline models by making the lower layers simpler than the upper layers, which we name as "the lower the simpler".


Besides, the gating operator also changes from the element-wise multiplication in GRU to the scalar multiplication * in SGU.

sGU is guaranteed to be the simplest among all the variants of GRU.


Besides, the forgetting factor α is designed as a hyper-parameter so that FOFE contains no trainable parameter.

fOfE is guaranteed to be the simplest among all the recurrent structures.


First, we assign a true/false label to each question-choice pair using the standalone QA system along with an associated confidence score s 1 .

we identify the natural language relation (entailment, contradiction or neutral) between each ordered pair of choices for a given question, along with an associated confidence score s 2 .


To make it more clear, bigger derivative/gradient indicates more impact and contribution toward the model's prediction.

the large-magnitude derivative values determine elements of input that would greatly affect f (x) if changed.


For each query q h i , we restrict its attention region (e.g., to a local scope with a fixed size M + 1 (M ≤ I) centered at the position i: Accordingly, the calculation of corresponding output in Equation 2is modified as: As seen, SANs are only allowed to attend to the neighboring tokens (e.g., K h , V h ), instead of all the tokens in the sequence (e.g., The SAN-based models are generally implemented as multiple layers, in which higher layers tend to learn semantic information while lower layers capture surface and lexical information (Peters et al., 2018;Raganato and Tiedemann, 2018).

we merely apply locality modeling to the lower layers, which same to the configuration in Yu et al.


Here, the predicted posterior mean itself is used in the target distribution.

this KL term does not force the model to learn the same posterior for all data samples (as in VAE), and thus, the decoder does not degrade to an unconditioned language model.


We see that DAE achieves the best BLEU score, which is not surprising because DAE directly optimizes the maximum likelihood of data as a surrogate of word prediction accuracy.

dAE performs poorly for probabilistic sentence generation as indicated by the other metrics.


We hypothesize that models that learn explicit alignments are particularly amenable to interpretable analysis on the task.

in addition to using the global attention model of (Luong et al., 2015), we adapt the transducer model proposed by Yu et al.


In contrast, qualitative analysis of the unparameterized transition model show that its alignments learn desirable correspondences (see Figure 2).

among our proposed models (trained on in-domain data only) the transducer with unparameterized transitions satisfies our desiderata of displaying both good predictive power for word generation, and learning interpretable alignments.


In any case, it is a statistical truism that negative results do not motivate acceptance of the null hypothesis.

it is possible that frequency ef-fects exist in naturalistic reading but are too small to be detected here.


However in practice, we have to make a trade-off between performances and the number of parameters (model complexity).

given that deep layers are more task-specific than shallow ones (Peters et al., 2018;Mou et al., 2016), and that word embeddings (shallow layers) contain the majority of parameters, we choose to expand only the top layers.


Furthermore, we have observed that despite the normalisation, the performances of the pre-trained classifiers were still much better than the randomly initialised ones.

to make them more competitive, we propose to start with optimising only the randomly initialised units while freezing the pretrained ones, then, launch the joint training.


This seems to be unreasonable in practice, especially for a long text segment such as a paragraph or a document, where the interaction between the first word and the last word is often negligible.

we address this limitation by proposing a local mixture of words, which tends to capture the semantic relations between neighboring words and undermine the long-range word dependencies.


Such new attempts highlight the breadth of commonsense phenomena, and make it evident that research on common sense has only scratched the surface.

there is need for datasets and models that will further our understanding of what is captured by current NLU models, and what are the main lacunae.


Finally, for each question, the worker chooses one additional distractor from CONCEPTNET, and authors another distractor manually.

in total, five candidate answers accompany each question.


Annotation artifacts in the Story Cloze Test, for example, allow models to achieve high performance while only looking at the proposed endings and ignoring the stories (Schwartz et al., 2017;Cai et al., 2017).

the development of benchmarks for common sense remains a difficult challenge.


Formulating questions for our task is nontrivial.

we only accept annotators for which at least 75% of the questions they formulate pass the verification process described below.


Unlike recent language representation models (Peters et al., 2018a; Radford et al., 2018), BERT is designed to pretrain deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers.

the pre-trained BERT model can be finetuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial taskspecific architecture modifications.


Some of the foundational papers in this area highlighted the limitations of trying to mitigate bias using methods that are "unaware" of protected attributes such as race, gender, or age (e.g., Dwork et al., 2012).

subsequent work has primarily focused on introducing fairness constraints, defined in terms of protected attributes, that reduce incentives to rely on undesirable correlations (e.g., Hardt et al., 2016;Zhang et al., 2018).


All the Phonetic Error Detection and Correction procedures will work on the word but it will be stored in a single dictionary.

distribution of words into the various dictionaries is crucial to the functioning of the system.


The oral proficiency interview has also been taken as an important validating criterion measure in the development of "indirect" standardized tests that intend to encompass an oral component, for example the TOEFL, TSE, and TOEIC tests from the Educational Testing Service.

all these tests rely, at least partially or indirectly, on the OPI oral proficiency construct.


When a person with limited proficiency in a language listens to a connected discourse (even on a familiar topic), much more time is spent in reconstructing what has been said, therefore the listener does not have as much time to consider the finer points of the message.

in both listening and speaking, if a person's control of the language is not automatic and immediate, there will likely be a corresponding decrement in the person's ability to use the language for a full range of communication tasks.


Speaking Style: The Repeat and Short Question items are written in a non-localized but colloquial style with contractions where appropriate.

a prompt will be written out (for the item speaker to recite) as, for example, "They're with the contractor who's late."


In the Reading and Repeat sections of the test, the responses consist of a complete phrase or sentence.

in addition to the accuracy of the response, we can also make use of the alignment and other properties of the speech signal to further judge the speaker's ability.


The subskill scores are based on distinct aspects of the candidates' performance and they are identified by names that reflect the performance criteria that they are intended to measure.

the Listening-Vocabulary score is derived only from items wherein the response task principally involves the immediate recognition of spoken words and ability to understand them and produce related words.


Although ba is a resultative verb, the resultative meaning does not reside in the lexical content.

in the matrix diagrams of ha, the resultative verb is lexically specified as `non-resultative' in SEMTYPE.


Nonetheless, this kind of implicature is beyond the limited intelligence of the parser.


But then if someone says 'are ga shiroi' (that is white), from our analysis of `ga' this forces 'are' to be in all `shiroi(x)' topics, notably in the singular topic of `kore' (thus forcing the identity `kore = are').

'x ga A' will contradict any 'y wa A' where y is singular and y x is already established.


So, in order to remove the ambiguity, we must add another clause that rules out the valuation of shiroi ('a) to any other object that is relevant in a kore-topic.

we feel the need for a `keredomo are wa shiroi ja nai' or `keredomo are wa kuroi' (if 'are' is relevant).


For example, in Japanese, (i) we can insert indeterminate expressions in the proposition, (ii) we can use disjunction to express indeterminacy of the truth of propositions as shown in (8a) and (8b), and (iii) we can use the disjunctive particle "ka" as sentence-final question marker to express indeterminacy of the truth of P, as shown in (8c).

we assume that question is essentially partitioning, which is expressed as disjunction in Japanese.


Along with (i) and (iii) and (vi) hold under the circumstances just described.

the tour guide knows that only one of the two gates is currently available, but not which one at a given moment.


The lower half of Figure 9 is intended to show that at the time when all the passengers have passed through the gate, the resulting distribution is partitioned into those who have passed through the west gate and those who have passed through the east gate.


The situation R has conjunctive potentials, west-gate-retrospective-potential and east-gate-retrospective-potential.

the situation P with west-gate-potential is accessible from R, while the situation Q with east-gatepotential is accessible from R, too.


Therefore, the situation P with west-gate-potential is accessible from R, while the situation Q with east-gatepotential is accessible from R, too.

pragmatically, that certain expression with disjunction obtains either "or"-reading or "and"-reading in a given situation reflects that only one of those readings can make the expression under consideration informative enough in that situation.


Here, if "sono kyaku (that passenger)" refers to a particular passenger and not a particular group of passengers, then the action that he/she took is singular.


an adequate theory of verbal semantics must include a theory of event structures.


It is uncontroversial that lexical information has to reflect various levels of representation in syntax as well as those of semantics and pragmatics.

the idea of MR in the lexicon is germane to the nature of the lexicon.


I The "done thing" role has to be identified with the embedded proposition, leaving the "object" role intact, which comes out as the OW argument.

the current lexicalist approach to word-formation makes it possible to bring semantic considerations to bear on the calculation of the argument structure of complex predicates.


The type of the content of the container koppu 'glass' is specified in the telic role of the qualia structure.

by assuming the elaborated lexicon, the constraint can be maintained, and it explains both the possible implicit targets and the impossible ones.


If the normal structure of the IHRC has an explicit target, and the one with an implicit target obtains as a result of coercion, it is not surprising that the qualia structure constrains the availability of the implicit target.

this discussion of this paper lends further support to the theory of GL and the qualia structure posited in it.


(28) Ul: kulayse incey ey cohci nayka kulay hanta hay kaciko hay therefore well oh good me so try determined Pollanikka talun kenun casinissnuntey swuhak-i mwunceyeyyo.

be_willing_to other thing convinced mathematics-NOM problem (I determined that I would try to do that.


Notice that Ul and U5 are in the same level of dialogue.

this case observes the discourse command constraint.


(27a) is a normal simple sentence, while (27b) is a multiple nominative sentence.


As a result, representation comes to play an important role of bridging the and computational processing.

representation comes to play an important role of bridging the gap between human and computer interactions.


These two argument roles shall be denoted by QDomain and Qrange.

in situation semantics, the propositional content of the linguistic meaning of the quantificational sentence EVERY STUDENT WALKS is expressed by the proposition: The situation s is supporting the quantificational information, while sj is where walking takes place.


an abstraction over QRange of the determiner relation every, where the QDomain role has been filled up by the type meaning of the noun.

the storage contains the pair (a, x 1 ) which consists of the semantical representation of the quantified NP and a parametric representative x 1 of QRange of every.


LPC states that reflexive generally prefers to seek its antecedent in the larger of the domains.

in the following example, the antecedent in the larger domain will be chosen as the best choice of antecedent for reflexive pronouns: John-top Bill-nom self-acc hate-comp believe 'John believes that Bill hates him.'


In (21b) where there is no sentential antecedent, DBC tells the grammar to search for a most prominent NP in the previous sentences, which is John in this case.

the sentence obtains a proper interpretation on the basis of discourse information.


In order to see how the rules in (5b) and (5c) interact, consider the case of topicalization here, as in 6 In (6), loves of category (S/LNP)/NP cannot combine with the object because the object Mary is preposed.

it has to combine with the subject John by function composition, which is type-raised into the function taking the verb phrase as argument.


Though the structures of lower clauses are different, the potential and tough constructions share the roperty that they allow any argument or adjunct of an embedded clause to be the matrix subject.

3 p as far as the nominative marking of a variety of arguments, they should involve the analogous process of derivations.