Interestingly, the concatenation approach appears to be competitive in this task, especially for the Spanish dataset, which appears to be better than the DARep approach.

we note such an approach does not capture any information transfer across different domains in the learning process.


However, we note such an approach does not capture any information transfer across different domains in the learning process.

our approach learns embeddings for the target domain by capturing useful cross-domain information and therefore can lead to improved modeling of embeddings that are shown more helpful for this specific down-stream task.


This makes both the information integration and stopping criteria welldefined.

in our focused reading domain, we do not know ahead of time which new pieces of information are necessarily relevant and must be taken in context.


Ozbal and Strapparava (2012) generate new words to describe a product given its category and properties.

their method is limited to handcrafted rules as compared to our data driven approach.


(2017) have proposed an approach to recommend brand names based on brand/product description.

they consider only a limited number of features like memorability and readability.


Unfortunately, practitioners who are qualified to diagnose and treat serious mental health issues such as schizophrenia are in chronically short supply, and their accumulated knowledge cannot be easily formalized into reproducible metrics (Patel et al., 2007).

clinical research into the symptoms and mechanisms of schizophrenia suggests that disturbances in language use, and especially in metaphor use and affect, characterize schizophrenia.


Hoax stories tend to use fewer superlatives and comparatives.

compared to other types of fake news, propaganda uses relatively more assertive verbs and superlatives.


By examining the results, we find an overall crossparty agreement of 46% regarding the discussed issues.

this agreement varies substantially if we consider the different macro-domains.


The possibility of measuring agreement at a finer level (topics) that is offered by our approach, shows, for example, that between 2004 and 2012 two opposite positions have been defined regarding the Middle East.

there has been a general agreement on the role of the U.S. concerning the relations with Europe.


These approaches extract concept and relation labels from syntactic structures and connect them to build a concept map.

common task definitions and comparable evaluations are missing.


Other types of information representation that also model concepts and their relationships are knowledge bases, such as Freebase (Bollacker et al., 2009), and ontologies.

they both differ in important aspects: Whereas concept maps follow an open label paradigm and are meant to be interpretable by humans, knowledge bases and ontologies are usually more strictly typed and made to be machine-readable.


Their approach can be interpreted as Q-BOT 'forgetting' the task after interacting with A-BOT.

this behavior of Q-BOT to remember the task only during dialog but not while predicting is somewhat unnatural compared to our setting.


Studies have shown that self expression and social support are beneficial in improving the individual's state of the mind (Turner et al., 1983;Choudhury and Kiciman, 2017) and thus such communities and interventions are important in suicide prevention.

there are often thousands of user posts published in such support forums daily, making it difficult to manually identify individuals at risk of self-harm.


Aside from the effort required to design effective features, these approaches usually model the problem with respect to the selected features and ignore other indicators and signals that can improve prediction.

our model only relies on text and is not dependent on any external or domain-specific features.


Previous selfreported diagnosis detection datasets contained a limited number of both control users and diagnosed users.

to this, we construct a new dataset with over 9,000 depressed users matched with a realistic number of control users.


Modern statistical learning approaches capture correlations among output variables in order to make coherent predictions.

for realworld applications, some implicit correlations are not appropriate, especially if they are amplified.


In practice, it is hard to obtain a solution where all corpus-level constrains are satisfied.

we show that the performance of the proposed approach is empirically strong.


The original dataset includes about 125,000 images with 75,702 for training, 25,200 for developing, and 25,200 for test.

the dataset covers many non-human oriented activities (e.g., rearing, retrieving, and wagging), so we filter out these verbs, resulting in 212 verbs, leaving roughly 60,000 of the original 125,000 images in the dataset.


It might be necessary to sacrifice some accuracy in order to satisfy privacy requirements.

this is not the case of all private information, since some of it is not relevant for the prediction of the text label.


In this paper we explore the following situation: (i) a main classifier uses a deep network to predict a label from textual data; (ii) an attacker eavesdrops on the hidden layers of the network and tries to recover information about the input text of unseen examples.

to previous work about neural networks and privacy (Papernot et al., 2016;Carlini et al., 2018) we do not protect the privacy of examples from the training set, but the privacy of unseen examples provided, e.g., by a user.


If its accuracy is high, then an eavesdropper can easily recover information about the input document.


In more details, for age, the adversary is well over the baseline in all cases except US.

gender seems harder to predict: the adversary outperforms the most frequent class baseline only in the +DEMO setting.


However, they only use a single adversary to alter the training of the main model and to evaluate the privacy of the representations, with the risk of overestimating privacy.

once the parameters of our main model are fixed, we train a new classifier from scratch to evaluate privacy.


These trends are persistent for all main-task/protected-attribute pairs we tried.

training the attacker network on the resulting encoder vectors reveals a different story.


6 We are interested in tweets whose protected attribute (race) is correctly predicted by the adversary.

at accuracy rates below 60%, many of the correct predictions could be attributed to chance.


Recent work on creating private representation in the text domain (Li et al., 2018) share our motivation of removing unintended demographic attributes from the learned representation using adversarial training.

they report only the discrimination accuracies of the adversarial component, and do not train another classifier to verify that the representations are indeed clear of the protected attribute.


Recent approaches counter this deficit by considering external sources related to a claim.

these methods require substantial feature modeling and rich lexicons.


2016 learned a part-of-speech (POS) tagger for their data and constructed a word level translation phrasebook to map emojis and slang to the Dictionary of Affect in Language (DAL) in order to identify their emotion.

to Blevins' translation approach, we leverage our large unlabeled dataset to automatically induce resources, such as word embeddings, that function well within the domain of our task.


2018b's research and reflects the fact that emotional states may fluctuate often and within a certain number of days.

word embeddings improved consistently as we extended the context window from 2 days to 90 days.


We expect our methods to be generalizable because we compute embeddings and lexicons from neighborhood-specific data and do not rely on large, hand-crafted resources such as dictionaries.

we hope to test generalizability in future work by applying our methods to other gang-related corpora, because there is variation in language, local concepts, and behavior across gangs.


2018), and NPN  -has focused on learning to predict individual entity states at various points in the text, thereby approximating the underlying dynamics of the world.

while these models can learn to make local predictions with fair accuracy, their results are often globally unlikely or inconsistent.


The commonsense constraints we have used for ProPara are general, covering the large variety of topics contain in ProPara (e.g., electricity, photosynthesis, earthquakes).

if one wants to apply ProStruct to other genres of procedural text (e.g., fictional text, newswire articles), or broaden the state change vocabulary, different commonsense constraints may be needed.


(2017) argue that Multi-NLI "[makes] it possible to evaluate systems on nearly the full complexity of the language."

how well does Multi-NLI test a model's capability to understand the diverse semantic phenomena captured in DNC?


The results are flipped on the two datasets focused on downstream tasks (Sentiment and RE) and MV.

the differences between pre-training on the DNC or Multi-NLI are small.


A first step toward grounded commonsense inference with today's deep learning machinery is to create a large-scale dataset.

recent work has shown that human-written datasets are susceptible to annotation artifacts: unintended stylistic patterns that give out clues for the gold labels (Gururangan et al., 2018;Poliak et al., 2018).


13 JOCI increases the scale by generating the hypotheses using a knowledge graph or a neural model.

to JOCI where the task was formulated as a regression task on the degree of plausibility of the hypothesis, we frame commonsense inference as a multiple choice question to reduce the potential ambiguity in the labels and to allow for direct comparison between machines and humans.


Wang (2017) release a larger dataset for fake news detection, and propose a hybrid neural network to integrate the statement and the speaker's meta data to do classification.

the presentation of evidences is ignored.


most existing models still have a number of drawbacks.


There is also work using deep learning methods to project different modality inputs into a common space, including restricted Boltzman machines (Ngiam et al., 2011;Srivastava and Salakhutdinov, 2012), autoencoders (Silberer and Lapata, 2014;Silberer et al., 2016), and recursive neural networks (Socher et al., 2013).

the above methods can only generate multimodal vectors of those words that have perceptual information, thus reducing multimodal vocabulary drastically.


Some recent work has investigated static image-based dialogue.

several real-world human interactions also involve dynamic visual context (similar to videos) as well as dialogue exchanges among multiple speakers.


(2014) used visual display information for on-screen item resolution in utterances for improving personal digital assistants.

we propose to employ dynamic video-based information as visual context knowledge in dialogue models, so as to move towards video-grounded intelligent assistant applications.


dog, sit, red) to the decoder instead of the image features, and is more effective in image captioning according to the evaluation on benchmark datasets.

the models based on conceptual information have a major drawback that it is hard for the model to associate the details with the specific objects in the image, because the visual words are inherently unordered in semantics.


The advantages are that the attention no longer needs to find the relevant generic regions by itself but instead find relevant bounding boxes that are object orientated and can serve as semantic guides.

the drawback is that predicting bounding boxes is difficult, which requires large datasets (Krishna et al., 2017) and complex models (Ren et al., 2015(Ren et al., , 2017a.


of approach can be seen as the extension of the earlier template-based slotting-filling approaches (Farhadi et al., 2010;Kulkarni et al., 2013).

few work studies how to combine the two kinds of attention models to take advantage of both of them.


α t ∈ R k is the attentive weights of V and the attentive visual input z t ∈ R g is calculated as The visual input z t and the embedding of the previous output word y t−1 are the input of the LSTM.

there is a noticeable drawback that the previous output word y t−1 , which is a much stronger indicator than the previous hidden state h t−1 , is not used in the attention.


ble 5, when extracting all words, providing more words to the model indeed increases the captioning performance.

even when top-20 all words are used, the performance is still far behind using only top-5 object words and seems to reach the performance ceiling.


Merging Gate Combing the visual attention and the topic attention directly indeed results in a huge boost in performance, which confirms our motivation.

directly combining them also causes lower scores in attributes, color, count, and size, showing that the advantages are not fully made use of.


With TEF, the performance of MCN can be significantly improved.

it is still inferior to our proposed TGN.


We test a state-of-the-art system (Peters et al., 2018) on PreCo and get an F1 score of 81.5.

a modest human performance (87.9, which will be described in 4.1 ) is much higher, verifying there remain challenges.


There are also specificities in each task.

existing methods for Chinese NER either do not exploit word boundary information from CWS or cannot filter the specific information of CWS.


In order to incorporate word boundary information from CWS task into NER task, Peng and Dredze (2016) propose a joint model that performs Chinese NER with CWS task.

their proposed model only focuses on task-shared information between Chinese NER and CWS, and ignores filtering the specificities of each task, which will bring noise for both of the tasks.


According to the results of Table 2 and Table 4, our proposed model achieves 4.67% and 1.43% improvement as compared with previous stateof-the-art methods on WeiboNER dataset and SighanNER dataset, respectively.

the overall performance on WeiboNER dataset is relatively low.


In this paper, we show that employing linguistic features in a neural coreference resolver significantly improves generalization.

the incorporated features should be informative enough to be taken into account in the presence of lexical features, which are very strong features in the CoNLL dataset.


Outputs of one model can also serve as features to the next model.

such an approach cannot model overlapping mentions of the same type, which frequently appear in practice.


In both of the previous approaches, their models would make local predictions and assign both "A" and "B" as left boundaries, and both "C" and "D" as right boundaries.

based on such local predictions one could also interpret "A B C" as a mention -this is where the ambiguity arises.


However, based on such local predictions one could also interpret "A B C" as a mention -this is where the ambiguity arises.

our model enjoys the structural ambiguity free property as it uses our newly defined I nodes (together with X nodes) to jointly capture the complete boundary information of mentions.


Since both variables are used to reconstruct the word, but only the y variable is trained to predict the tag, it appears that z is capturing other information useful for reconstructing the word.

since they are both used for reconstruction, the two spaces show signs of alignment; that is, the "tag" latent variable y does not show as clean a separation into tag clusters as the y variable in the VSL-GG-Hier model in Figure 3e.


These methods focused on mono-lingual settings.

for cross-lingual tasks (e.g., cross-lingual entity linking), these approaches need to introduce additional tools to do translation, which suffers from extra costs and inevitable errors (Ji et al., , 2016.


Thus, monolingual structured knowledge of entities are not only extended to cross-lingual settings, but also augmented from other languages.

we utilize distant supervision to generate comparable sentences for cross-lingual sentence regularizer to model co-occurrence information across languages.


Moens (2015, 2016) collect comparable documents on same themes from multi-lingual Wikipedia, shuffle and merge them to build "pseudo bilingual documents" as training corpora.

the quality of "pseudo bilingual documents" are difficult to control, resulting in poor performance in several cross-lingual tasks (Vulic and Moens, 2016).


On one hand, we utilize their shared semantics to align similar words and entities with similar embedding vectors, no matter they are in the same language or not.

crosslingual embeddings will benefit from different languages due to the complementary knowledge.


Conventional knowledge representation methods normally regard cross-lingual links as a special equivalence type of relation between two entities (Zhu et al., 2017).

we argue that this may mislead to an inconsistent training ob-multiple relations.


Another approach in (Han et al., 2016;Toutanova et al., 2015;Wu et al., 2016) learns to represent entities based on their textual descriptions together with the structured relations.

these methods only focus on mono-lingual settings, and few researches have been done in cross-lingual scenarios.


we argue that this may mislead to an inconsistent training ob-will be no direct relation between Chinese entity 福 斯特 (Foust) and English entity Piston by merely adding the equivalence relation between Foust and 福斯特, which is in contradiction with the fact tha Foust belongs to Piston, no matter in which lan guage.


In this paper we consider both options, taking advantage of the second in order to feed the model with BEs.

to standard LSTM-based language  Figure 1: The AE-SCL and AE-SCL-SR models (figure imported from ZR17).


We focus on learning linear mappings to construct the common semantic space and adopt correlational neural networks (CorrNet) (Chandar et al., 2016;Rajendran et al., 2015) as the basic model.

to previous work which only exploited monolingual word semantics, we introduce multiple cluster-level alignments and design a new cluster consistent CorrNet to align both words and clusters.


Furthermore, recent work proposes approaches to obtain unsupervised BWEs without relying on any bilingual resources (Zhang et al., 2017;Lample et al., 2018b).

to BWEs that only focus on a pair of languages, MWEs instead strive to leverage the interdependencies among multiple languages to learn a multilingual embedding space.


In the crosslingual setting, it has been successfully applied to unsupervised cross-lingual text classification (Chen et al., 2016) and unsupervised bilingual word embedding learning (Zhang et al., 2017;Lample et al., 2018b).

these methods only consider one pair of languages at a time, and do not fully exploit the cross-lingual relations in the multilingual setting.


In CLWS though, one can still achieve relatively high correlation in spite of minor inaccuracies.

an encouraging result is that when compared to the state-of-the-art supervised results, our MAT+MPSR method outperforms NASARI by a very large margin, and achieves top-notch overall performance similar to the competition winner, Luminoso, without using any bitexts.


2017leveraged cross-lingual signals in more than two languages.

they either used pretrained embeddings or learned only for the English side, which is undesirable since cross-lingual embeddings shall be jointly learned such that they aligned well in the embedding space.


(2012) presented SCWS, the first and only dataset that contains word pairs and their sentential contexts for measuring the quality of sense embeddings.

it is a monolingual dataset constructed in English, so it cannot evaluate cross-lingual semantic word similarity.


This procedure, called post-specialization, effectively propagates the information stored in the external constraints to the entire word vector space.

this mapping should not just model the inherent transformation, but also ensure that the resulting vector is 'natural'.


We observe only modest and inconsistent gains over ATTRACT-REPEL and POST-DFFN in the FULL setting.

the explanation of this finding is straightforward: 99.2% of SimLex words and 99.9% of SimVerb words are present in the external constraints, making this an unrealistic evaluation scenario.


Because of this, monolingual embedding spaces are not isomorphic Kementchedjhieva et al., 2018).

simply dropping the orthogonality constraints leads to overfitting, and is thus not effective in practice.


Some common signals to learn bilingual embeddings come from parallel (Hermann and Blunsom, 2014;Luong et al., 2015;Levy et al., 2017) or comparable corpora (Vulić and Moens, 2015a;Søgaard et al., 2015;Vulić and Moens, 2016), or lexical resources such as WordNet, ConceptNet or BabelNet (Speer et al., 2017;Mrksic et al., 2017;Goikoetxea et al., 2018).

these sources of supervision may be scarce, limited to certain domains or may not be directly available for certain language pairs.


Furthermore, one may wonder whether the initial alignment is actually needed, since e.g., Coates and Bollegala (2018) obtained high-quality meta-embeddings without such an alignment set.

when applying our approach directly to the initial monolingual non-aligned embedding spaces, we obtained results which were competitive but slightly below the two considered alignment strategies.


In Italian our proposed model shows an improvement across all configurations.

in Spanish VecMap emerges as a highly competitive baseline, with our model only showing an improved performance when training data in this language abounds (in this specific case there is an increase from 17.2 to 19.5 points in the MRR metric).


Answering this directly would require annotators with domain expertise and is infeasible in practice.

we can use our crowdsourced annotation to answer a restricted variant of this question: given a sentence s and an insertable phrase p, do humans agree on where p belongs in s?


A key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language.

this ambition is largely hampered by the variation in structural and semantic properties, i.e.


Deep learning has allowed NLP algorithms to dispose of manually-crafted features, and to virtually achieve language independence.

their performance still varies noticeably across languages due to different underlying data distributions (Bender, 2013;O'Horan et al., 2016).


The rationale behind fixing the set V is a) to make the language model more robust to handling OOVs and to effectively bypass the problem of unreliable word estimates for low-frequency and unseen words (by ignoring them), and b) to enable direct comparisons of absolute perplexity scores across different models.

this posits a critical challenge as cross-linguistic evaluation becomes uneven.


This setup is commonly referred to as the open-vocabulary setup.

two distinct approaches with crucial modeling differences are referred to by the same term in the literature.


Unseen test words are mapped to one <UNK> vector, sampled from the the space of trained word vectors relying on a normal distribution and the same fixed random seed for all models.

kN5 by design has a slightly different way of handling unseen test words: they are regarded as outliers and assigned low-probability estimates.


We have also observed that injecting character information into word representations is always beneficial because this mitigates the above-mentioned sparsity issues.

the extent of the gain in perplexity partly depends on some typological properties that regulate the ambiguity of the mapping between morphemes (here modeled as character n-grams) and their meaning.


Others have focused on token-level language ID; some work is constrained to predicting word-level labels from a single language pair (Nguyen and Dogruöz, 2013;Solorio et al., 2014;Molina et al., 2016a;Sristy et al., 2017), while others permit a handful of languages (Das and Gambäck, 2014;Sristy et al., 2017;Rijhwani et al., 2017).


Models like Skip-gram, CBOW (Mikolov et al., 2013a) and GloVe (Pennington et al., 2014) are immensely popular and achieve remarkable performance in many NLP tasks.

most WRL methods learn distributional information of words from large corpora while the valuable information contained in semantic lexicons are disregarded.


Another way of performing language independent transfer resorts to multi-task learning, where a model is trained jointly across different languages by sharing parameters to allow for knowledge transfer (Ammar et al., 2016a;Cotterell and Duh, 2017;Lin et al., 2018).

such approaches usually require some amounts of training data in the target language for bootstrapping, which is different from our unsupervised approach that requires no labeled resources in the target language.


Beam search is a widely used approximate search strategy for neural network decoders, and it generally outperforms simple greedy decoding on tasks like machine translation.

this improvement comes at substantial computational cost.


These decoders generate tokens from left to right, at each step giving a distribution over possible next tokens, conditioned on both the input and all the tokens generated so far.

since the space of all possible output sequences is infinite and grows exponentially with sequence length, heuristic search methods such as greedy decod-ing or beam search (Graves, 2012;Boulanger-Lewandowski et al., 2013) must be used at decoding time to select high-probability output sequences.


Neural machine translation (NMT) based on the encoder-decoder architecture becomes the new state-of-the-art due to distributed representation and end-to-end learning (Cho et al., 2014;Bahdanau et al., 2015;Junczys-Dowmunt et al., 2016;Gehring et al., 2017;Vaswani et al., 2017).

the current NMT is a global model that maximizes the performance on the overall data and has problems in handling low-frequency words and ambiguous words 1 , we refer these words as troublesome words and define them in Section 3.1.


Therefore, we have to leverage external bilingual data sources to alleviate the problem.

the external bilingual data are usually not in the same domain as the utterance, and hence they are not aligned with the slot-value pair and system acts (i.e., (c e s , c e v , a e t ) or (c f s , c f v , a f t )).


From the table Table 3, we can observe that the α ablation (τ fixed to 0.1) τ ablation (α fixed to 1)  experimental results are not very sensitive to α, a dramatic change of α will not harm the final results too much, we simply choose α = 1 as the hyper-parameter.

the system is more sensitive to temperature.


The language embeddings need to represent all of the languagespecific information and thus may need to be large in size.

when computing the parameters of each group, only a small part of that information is relevant.


Low-Resource: Similar to the supervised experiments except that we limit the size of the parallel corpora used in training.

for GML and CPG the full monolingual corpus is used for auto-encoding training.


This means that, in the previous example, we would first translate from German to English and then from English to French (using two pairwise models for a single translation).

pivoting is prone to error propagation incurred when chaining multiple imperfect translations.


Our resulting system, which outperforms other state-of-the-art systems, uses a standard pairwise encoder-decoder architecture.

it differs from earlier approaches by incorporating a component that generates the parameters to be used by the encoder and the decoder for the current sentence, based on the source and target languages, respectively.


One can see that all systems perform similarly in the beginning and converge after observing increasingly more training instances.

the model with the ratio of (1:10) synthetic data gets increasingly biased towards the noisy data after 1M instances.


Above we examined the mean of prediction loss for each token over all occurrences, in order to identify difficult-to-predict tokens.

the uncertainty of the model in predicting a difficult Input: Difficult tokens and the corresponding sentences in the bitext D " tyt, Yy t " ry1, .


As expected using random sampling for backtranslation improves the translation quality over the baseline.

all targeted sampling variants in turn outperform random sampling.


This is inherently useful since it allows for better learning of less frequent words.

a side effect of this approach is that at times the model generates subword units that are not linked to any words in the source sentence.


In recent years, neural machine translation (NMT) has achieved great advancement (Nal and Phil, 2013;Sutskever et al., 2014;Bahdanau et al., 2015).

two difficulties are encountered in the practical applications of NMT.


In this way, the domain-shared translation knowledge can be fully exploited.

the translated sentences often belong to multiple domains, thus requiring a NMT model general to different domains.


It should be noted that our utilization of domain classifiers is similar to adversarial training used in (Pryzant et al., 2017) which injects domain-shared contexts into annotations.

by contrast, we introduce domain classifier and adversarial domain classifier simultaneously to distinguish different kinds of contexts for NMT more explicitly.


datasets of paired sentences in both the source and target language.

bitext is limited and there is a much larger amount of monolingual data available.


This is likely because synthetic beam and greedy data does not provide as much training signal as the bitext which has more variation and is harder to fit.

sampling and beam+noise require no upsampling of the bitext, which is likely because the synthetic data is already hard enough to fit and thus provides a strong training signal ( §5.2).


(Sproat et al., 2006;Chang et al., 2009) is considerably easier than generation, owing to the smaller search space.

discovery often uses features derived from resources that are unavailable for low-resource languages, like comparable corpora (Sproat et al., 2006;Klementiev and Roth, 2008).


Inducing multilingual word embeddings by learning a linear map between embedding spaces of different languages achieves remarkable accuracy on related languages.

accuracy drops substantially when translating between distant languages.


Fixed Decoding Depth {2,3,4,5}-pass decoders perform the left-to-right decoding by the multipass decoder with a fixed number of decoding passes.

to the related machine translation systems, our fixed number-pass decoder significantly outperforms Moses and RNNSearch by 7.53 and 1.05 BLEU points at least, as Table 2 presents.


Specifically, the adaptive multi-pass decoder outperforms the multi-pass decoder with a fixed decoding depth by 0.69, 0.71, 0.68 and 0.45 BLEU scores on NIST03, NIST04, NIST05 and NIST06 datasets at most.

to the Moses, RNNSearch, Deliberation Network and ABDNMT, the adaptive multi-pass decoder has the corresponding improvement about 8.03, 1.55, 0.74 and 0.34 BLEU points, respectively.


In training phrase, we spend more time training the multipass decoder than RNNSearch, Deliberation Network and ABDNMT.

in testing phrase, as illustrated in Table 2, our adaptive multi-pass decoder spends about 180s completing the entire testing procedure, in comparison with the corresponding 87s, 162s, 132s of RNNSearch, Deliberation Network and ABDNMT, due to the auxiliary policy network.


While conventional machine comprehension models were given a paragraph that always contains an answer to a question, some researchers have extended the models to an open-domain setting where relevant documents have to be searched from an extremely large knowledge source such as Wikipedia (Chen et al., 2017;Wang et al., 2017a).

most of the open-domain QA pipelines depend on traditional information retrieval systems which use TF-IDF rankings (Chen et al., 2017;Wang et al., 2017b).


Despite the efficiency of the traditional retrieval systems, the documents retrieved and ranked at the top by such systems often do not contain answers to questions.

simply increasing the number of top ranked documents to find answers also increases the number of irrelevant documents.


Estimating θ a and θ g is straight-forward by using the cross-entropy objective J 1 ({θ a , θ g }) and the backpropagation algorithm.

selecting text regions in the Context Zoom layer makes it difficult to estimate θ z given their discrete nature.


The cosine similarity of the centroids of the two versions of Hyperwords is -0.006, and the cosine similarity for Hyperwords-SVD and GloVe is 0.019.

poor initialization as a result of applying the identity transform to very distant word embeddings is not the explanation for the poor performance of MUSE in this set-up: Both sets of Hyperwords embeddings were normalized, but alignment still failed.


In all the word pairs, average word frequency is 13934.4.

it is only 1676.1 in the top 0.1% word pairs, it is 3984.8 in the top 1%, and it is 7904.9 in the top 10%.


Word embeddings have been an essential part of neural-network based approaches for natural language processing tasks (Goldberg, 2016).

many popular word embeddings techniques have a fixed vocabulary (Mikolov et al., 2013;Pennington et al., 2014), i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus.


To compensate for the absence of direct supervision, work in crosslingual learning and distant supervision has discovered creative use for a number of alternative data sources to learn feasible models: -aligned parallel corpora to project POS annotations to target languages (Yarowsky et al., 2001;Agić et al., 2015;Fang and Cohn, 2016), -noisy tag dictionaries for type-level approximation of full supervision (Li et al., 2012), -combination of projection and type constraints (Das and Petrov, 2011;Täckström et al., 2013), -rapid annotation of seed training data .

only one or two compatible sources of distant supervision are typically employed.


Europarl covers 21 languages of the EU with 400k-2M sentence pairs, while WTC spans 300+ widely diverse languages with only 10-100k pairs, in effect sacrificing depth for breadth, and introducing a more radical domain shift.

as our results show little projected data turns out to be the most beneficial, reinforcing breadth for depth.


All data sources employed in our experiment are very high-coverage.

for true low-resource languages, we cannot safely assume the availability of all disparate information sources.


(ii) Unlike the adversarial methods, our accuracy generally mirrors the model's loss.

the various losses of the adversarial approaches do not well reflect translation accuracy, making model selection or early stopping a challenge in itself.


Another idea is to replace the softmax in soft attention with sparsity inducing operators (Martins and Astudillo, 2016; Niculae and Blondel, 2017).

all sparse/local attention methods continue to compute P (y) from an attention weighted sum of inputs (Eq: 2) unlike hard attention.


Our beam-joint also is a mixture of softmax and possibly achieves higher rank than a single softmax.

their mixture requires learning multiple softmax matrices, whereas ours are due to varying attention and we do not learn any extra parameters than soft attention.


For example, pessimism and negative attitudes impact negatively one's mental health, can induce suicidal thoughts, and affect negatively not only the person in question, but also their family and friends (Peterson and Bossio, 2001;Achat et al., 2000;Scheier et al., 2001).

optimism reduces stress and promotes better physical health and overall well-being (Carver et al., 2010).


Our main focus in this paper is on MTL as a framework to explore the lexical, structural and topical knowledge involved in users' selection of headlines.

recognizing a popular headline and giving advice on how to write one are not the same: we want to provide editors and journalists with insights as to what constructions are likely to attract more eyeballs.


Menini and Tonelli (2016) develop a SVM classifier to detect disagreement, relying on three aspects including sentiment-based, semantic and surface features extracted from both whole text and topic-related part.

the performances of all these models highly depend on the quality of hand-crafted features.


in response, which indicates a rhetorical mood to show disagreement.

even though why doesn't he answer in response is endowed less weight from the self attention, the cross attention highlights it and god in quote.


We can only retrofit the embeddings of authors in the training set D train , since we need information about the class label in order to construct Ω.

the retrofitting process changes the configuration of the embedding space (intoD train ), so a separating hyperplane learned onD train will not be applicable to a test set D test in the original embedding space.


Research has identified a variety of linguistic features, ranging from "stylistic features with n-grams models, parts-of-speech, collocations, LDA, different readability indexes, vocabulary richness, correctness or verbosity" (Rangel et al., 2016).

none of these papers used demographic information directly in the author representations.


(1995) proposed a semisupervised framework active learning, which can minimize the need for human annotation in a certain extent.

these approaches are still mostly or selectively human-labeled, and may be distracted by the disadvantages raised above.


To train the GAN, we used the distributed sentence representations computed from the pretrained sentence embeddings instead of symbolic sentences.

we think the limitation of the pre-trained sentence embeddings can be overcome by building a GAN that generates symbolic sentences and discriminates them.


It is noteworthy that AL is not an "once-for-all" deal.

it needs to be carried out repeatedly and iteratively, until a predefined condition is met, such as the time at which the classification performance remains almost the same within  We cooperate AL with Exp2Imp transformation.


• This makes it possible to learn the subtle differences among argument pairs which belong to different relation classes.

the instances listed below are far from informative.


Neural networks based models like Seq2Seq architecture (Vinyals and Le, 2015;Shang et al., 2015) are proven to be effective to generate valid responses for a dialogue system.

as revealed in many previous works (Li et al., 2016a;Wu et al., 2018), "safe reply" is still an open problem and lots of efforts are made to generate more informative responses (Li et al., 2016a;Mou et al., 2016;Li et al., 2016b;Qiu et al., 2017;He et al., 2017;.


Note that the size of training set and vocabulary used in our experiments are relatively small compared to millions of qa-pairs used in other works Wu et al., 2018), so it's reasonable that bad cases sometimes occur in results of baselines.

our models, no matter the static one or the dynamic one, could generate amazing responses which are not only grammatical and informative, but also have some emotional expressions like the use of punctuation and repetition.


Recently, Reinforcement Learning (RL) approaches have demonstrated advanced performance in image captioning by directly optimizing the metric used for testing.

this shaped reward introduces learning biases, which reduces the readability of generated text.


Actor-critic (Sutton and Barto, 1998) methods are often adopted, which involves training an additional value network to predict the expected reward.

(Rennie et al., 2017) designed a self-critical method that utilizes the output of its own test-time inference algorithm as the baseline to normalize the rewards, which leads to further performance gains.


The WordNet graph has edges of various types, with the main types being hypernymy and meronymy to connect nodes containing senses.

we do not use these types, and consider an edge as an undirected semantic or lexical relation between two synsets.


For example, consider the following connected nodes: The PPR vectors of suit and dress have some weight on tailor, which is desirable.

the PPR vector of law will also have a non-zero weight for tailor.


In this paper we use the same principle and reward n-grams that are found in the source document during the AMRto-Text generation process.

we use a simpler approach using a probabilistic language model in the scoring mechanism.


We hypothesize this can be attributed to how the AMR dataset is annotated as there might be discrepancies in different annotator's choices of AMR concepts and relations for sentences with similar wording.

the AMR parsers introduce errors, but they are consistent in their choices of AMR concepts and relations.


For that, the earliest Document Understand Conference (DUC) (NIST, 2011) benchmarks, in 2001 and 2002, defined several target summary lengths and evaluated each summary against (manually written) reference summaries of the same length.

due to the high cost incurred, subsequent DUC and TAC (NIST, 2018) benchmarks (2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014), as well as the more recently popular datasets CNN/Daily Mail (Nallapati et al., 2016) and Gigaword (Graff et al., 2003), included references and evaluation for just one summary length per input text.


EXTRACT also outperforms previously published extractive models (i.e., SummaRuNNer, EXTRACT-CNN, and REFRESH).

note that SummaRuNNer generates anonymized summaries (Nallapati et al., 2017) while our models generate non-anonymized ones, and therefore the results of EXTRACT and SummaRuNNer are not strictly comparable (also note that LEAD3 results are different in Table 1).


Among the aforementioned related studies, a few proposed systems explicitly targeted at generating abstractive summaries for documents.

these systems highly rely on the attention mechanism and/or copying mechanism that heavily depends on different part of input during the decoding stage.


Short text categorization is widely studied since the recent explosive growth of online social networking applications (Song et al., 2014).

with documents, short texts are less topic-focused in texts.


Major attempts to tackle the problem is to expand short texts with knowledge extracted from the textual corpus, machine-readable dictionaries, and thesauri (Phan et al., 2008;Wang et al., 2008;Chen et al., 2011;Wu et al., 2012).

because of domain-independent nature of dictionaries and thesauri, it is often the case that the data distribution of the external knowledge is different from the test data collected from some specific domain, which deteriorates the overall performance of categorization.


This approach uses document labels to influence the selection of anchor words, which in turn affects the resulting topics.

supervised Anchors requires a downstream classifier to be trained using topics as features.


Our algorithm, called Labeled Anchors, also augments the vector-space representation to include the L document labels.


Therefore, the process of building a classifier scales linearly with the number of documents and can be time consuming compared to topic recovery.

the formulation of Labeled Anchors allows us to construct a classifier with no additional training.


Intuitive, high frequency words usually appear in multiple topics.

if we examine Equation 2, we can see that the anchor words are just points in V -dimension space; they do not actually have to correspond to any particular word so long as that point in space uniquely identifies a topic.


With just baseline anchors from Gram-Schmidt, the classification accuracy of Labeled Anchors is on par with that of Supervised Anchors using logistic regression as the downstream classifier.

because Labeled Anchors is fast enough to allow interaction, participants are able to improve classification accuracy on the development set by an average of 5.31%.


In topic modeling so far, perplexity is a direct optimization target.

topic coherence, owing to its challenging computation, is not optimized for and is only evaluated after training.


Among all datasets, we observed improved NPMI at the same perplexity level, validating the effectiveness of the topic coherence regularization.

on the NYTimes dataset, the improvement is quite marginal even though WETC improvements are very noticeable.


More recently, augmenting this neural architecture with the attention mechanism Luong et al., 2015) has dramatically increased the quality of results across most NLP tasks.

in text normalization, state-of-the-art results involving attention (e.g., Xie et al.


Arabic diacritization, which can be considered forms of text normalization, has received a number of neural efforts (Belinkov and Glass, 2015;Abandah et al., 2015).

state-of-the-art approaches for end-to-end text normalization rely on several additional models and rule-based approaches as hybrid models (Pasha et al., 2014;Nawar, 2015;Zalmout and Habash, 2017), which introduce direct human knowledge into the system, but are limited to correcting specific mistakes and rely on expert knowledge to be developed.


While tuning scheduled sampling, we found that introducing a sampling probability provided better results than relying on the ground truth, i.e., teacher forcing (Williams and Zipser, 1989).

introducing a schedule did not yield any improvement as opposed to keeping the sampling probability constant and unnecessarily complicates hyperparameter search.


In image processing, simple augmentation techniques such as flipping, cropping, or increasing and decreasing the contrast of the image are both widely utilized and highly effective (Huang et al., 2016;Zagoruyko and Komodakis, 2016).

it is nontrivial to find simple equivalences for NLP tasks like machine translation, because even slight modifications of sentences can result in significant changes in their semantics, or *: Equal contributions.


Word-by-word translation always outputs a target word for every position.

there are a plenty of cases that multiple source words should be translated to a single target word, or that some source words are rather not translated to any word to make a fluent output.


Neural LMs globally score the entire candidate plaintext sequence (Mikolov et al., 2010).

using a neural LM for decipherment is not trivial because scoring the entire candidate partially deciphered plaintext is computationally challenging.


Greydanus (2017) frames the decryption process as a sequence-to-sequence translation task and uses a deep LSTM-based model to learn the decryption algorithms for three polyalphabetic ciphers including the Enigma cipher.

this approach needs supervision compared to our approach which uses a pre-trained neural LM.


Previous work on using context for sentence classification used LSTM and CNN network layers to encode the surrounding context, giving an improvement in classification accuracy (Lee and Dernoncourt, 2016).

the use of CNN and LSTM layers imposes a significant computational cost when training the network, especially if the size of the context is large.


More traditional approaches employ an off-line structure predictor (e.g., a parser) to define the computation graph (Tai et al., 2015;Chen et al., 2017), sometimes with some parameter sharing (Bowman et al., 2016).

these off-line methods are unable to jointly train the latent model and the downstream classifier via error gradient information.


1 Code available at https://github.com/ sweetpeach/ReCode Tree-based approaches (Yin and Neubig, 2017;Rabinovich et al., 2017) represent code as Abstract Syntax Trees (ASTs), which has proven effective in improving accuracy as it enforces the well-formedness of the output code.

representing code as a tree is not a trivial task, as the number of nodes in the tree often greatly exceeds the length of the NL description.


Word n-grams are obvious candidates when generating a sequence of words as output, as in NMT.

in syntax-based code generation, the generation target is ASTs with no obvious linear structure.


(2016) proposes a sequence-to-sequence (Seq2Seq) network to model the SQL query and natural language jointly.

since the SQL is designed * Work done when the author was at IBM Research.


Most existing works of graph convolution neural networks focus more on node embeddings rather than graph embeddings (GE) since their focus is on the node-wise classification task.

graph embeddings that convey the entire graph information are essential to the downstream decoder, which is crucial to our task.


One possible reason is that in our graph encoder, the node embedding retains the information of neighbor nodes within K hops.

in the tree encoder, the node embedding only aggregates the information of descendants while losing the knowledge of ancestors.


CDMM assigns a high attention value to the word to predict Order and Accept compared to Discuss.

the attention values differ according to the speaker.


Accept as compared to the other classes when the speaker is king.

when officials use this word, CDMM assigns a high attention value to the word in the Order class.


Table 4 demonstrates the performance of our model over different window size K. We can see that all these results is better than the performance our model without attention mechanism.

a proper restriction window is helpful for the attention mechanism to take better effect.


Video content on social media platforms constitutes a major part of the communication between people, as it allows everyone to share their stories.

if someone is unable to consume video, either due to a disability or network bandwidth, this severely limits their participation and communication.


Feature-modulating methods like FiLM (De Vries et al., 2017;Perez et al., 2018) control image-comprehension process using modulation-parameters generated from the question, allowing models to be trained end-to-end.

the image-comprehension program in visual reasoning tasks can be extremely long and sophisticated.


The result shows that visual features can guide the comprehension of question logics with textual modulation.

question-based modulation parameters enable the ResBlocks to filter out irrelative objects.


More importantly, most state-of-the-art systems can only predict one most likely relation for a single entity pair.

it is very common that one sentence may contain multiple entity pairs and describe multiple relations.


To filter such strings, we need to rely on indicators that may reside in a different line such as a heading "Dissertations supervised".

if we only train the model on webpage-level input, the model may be dominated by the longest line on the homepage, such as biography information.


First, we inject character information into the model through character-level CNNs; these give the model a deeper ability to recognize character correspondences between the context and entity title.

these convolutional filters struggle to learn useful features in this noisy context and ultimately do not help performance.


Think The Warriors The gold title is The Warrior (film) and the base model correctly places 90% of its attention weight on the word movie when calculating attention for this title.

the character-level CNN model only places 60% of its attention weight on it, distributing its attention values more evenly across the rest of the words.


Fact salience is close to automatic text summarization (Erkan and Radev, 2004); both must detect the most prominent information in the text.

while text summarization generates summaries for humans, fact salience output must be interpretable by machines.


As finding and annotating such potential duplicates manually is very tedious and costly, automatic methods based on machine learning are a viable alternative.

many forums do not have annotated data, i.e., questions labeled by experts as duplicates, and thus a promising solution is to use domain adaptation from another forum that has such annotations.


We can see that the Wasserstein and the classification-based methods perform very similarly, after proper hyper-parameter tuning.

wasserstein yields better stability, achieving an AUC variance 17 times lower than the one for classification across hyper-parameter settings.


Several question answering datasets have been proposed (Berant et al., 2013;Joshi et al., 2017;Trischler et al., 2017;Rajpurkar et al., 2018, inter alia).

all of them were limited to answering individual questions.


(2018) study the problem of sequential question answering, and introduce a dataset for the task.

we differ from them in two aspects: 1) They consider question-answering over structured knowledge-bases.


In the second iteration, a number of test samples (to which the classifier associated higher confidence scores) are added to the training set for another round of training.

the ground-truth labels of the added test samples are not necessary.


Therefore, we propose a shared-private (SP) model as shown in Fig 1.c, where we employ a shared LSTM layer to extract shared sentiment features for both sentiment and emotion classification tasks, and a target-specific LSTM layer to extract specific emotion features that are only sensitive to our emotion classification task.


The task of sentiment modification requires reversing the sentiment of the input and preserving the sentiment-independent content.

aligned sentences with the same content but different sentiments are usually unavailable.


For instance, when the source text is "This is a wonderful movie", we expect an output like "This movie is disappointing".

the generated sentence may be "The waiters are very rude", which has little relevance to the source text.


(2017) augment the unstructured variables z in vanilla VAE with a set of structured variables c each of which targets a salient and independent semantic feature of sentences, to control sentence sentiment.

all of these work attempt to implicitly separate the non-emotional content from the emotional information in a dense sentence representation.


The main reason is that these methods using adversarial learning attempt to implicitly separate the emotional information from the context information in a sentence vector.

without parallel data, it is difficult to achieve such a goal.


This approach allows the system to learn from data sets across multiple domains, increasing the flexibility of the sentiment classifier.

the weakness of this approach is that the system uses a bigram bag-of-words as input, making it unable to learn long-distance syntactic phenomena.


We note a limitation of our experiment: the selection of arbitrary adjectives.

the 15 adjectives used are commonly used to describe humans, and the result was quite consistent.


Similar to the FakeNews, (Rashkin et al., 2017;Vlachos and Riedel, 2014) focused on political statements from Politifact.com to verify the degree of truthfulness.

they assume that the gold standard documents containing the evidence are already known, which overly simplifies the task.


Our DA rte model must correctly decide whether a claim is NEI, when the evidence retrieved is irrelevant and insufficient.

nEI claims have no annotated evidence, thus cannot be used to train RTE.


NoScoreEv is a simple classification accuracy that only considers the correctness of the verification label.

scoreEv is a stricter measure that also considers the correctness of the retrieved evidence.


The number of these actions is regarded as a measure of popularity.

popularity is not determined solely by content of a post, e.g., a text or an image it contains, but is highly based on its contexts, e.g., timing, and authority.


Categories of news articles (Isonuma et al., 2017) and ratings of online reviews (Xiong and Litman, 2014) were used as distant labels in extractive summarization.

these have been used as supplementary labels to enhance conventional summarization models, whereas we present labels which a model can solely be trained with.


We assume that measures of popularity reflects the informativeness, the index required for a summary (Erkan and Radev, 2004), and validate whether popularity can be used as a distant label for extractive summarization.

popularity is not solely determined by content, e.g., a text or an image, but is highly affected by contexts, e.g., timing, and authority (Cheng et al., 2017;Burghardt et al., 2017;Suh et al., 2010;Hessel et al., 2017;Jaech et al., 2015).


(Rotter, 1966) to identify an external locus of control when authors express that they feel controlled by other people or the environment.

authors communicate an internal locus of control when they ascribe the control of their decisions and circumstances to themselves.


Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences.

we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation.


Moreover, "a decade as" appears 2,280 times, but "its defeat is" appears only 24 times.

this is undesirable because if there is another hypothesis that happens to be the same as reference, which will not be ranked as the best candidate.


All methods reduce the WER over the baseline without rescoring.

lMlM and rlMlM are notably better than the other two methods.


Both of them demonstrate superior performance over their minimum-PPL counterparts.

the performance gain from LMLM to rLMLM is small, although rLMLM is built on more pairwise comparisons and requires more training efforts.


These pre-trained layers brought significant improvements to various NLP benchmarks, yielding  up to 30% relative error reductions.

due to high variability of language, gigantic NNs (e.g., LSTMs with 8,192 hidden states) are preferred to construct informative LMs and extract multifarious linguistic information (Peters et al., 2017).


Hence, we propose to compress the model by layer selection, which retains useful layers for the target task and prunes irrelevant ones.

for the widely-used stacked-LSTM, directly pruning any layers will eliminate all subsequent ones.


Since the training of language models needs nothing but the raw text, it has almost unlimited corpora.

conducting training on extensive corpora results in a huge dictionary, and makes calculating the vanilla softmax intractable.


The ideal choice for R would be the L 0 regularization of z, i.e., R 0 (z) = |z| 0 .

it is not continuous and cannot be efficiently optimized.


The model significantly outperforms a strong "Frequency" baseline in our experiments.

there are other discourse relations beyond lexical similarity.


Adding each voting feature individually produces mixed results.

adding all voting features improves all metrics.


2. it introduces weighted soft count kernels.

the PageRank baseline also does embedding tuning but produces poor results, thus the second change should be crucial.


The models predict a starting point (s) and duration (d) for each given temporal entity (t 1 , e 1 , and t 2 ) in the input.

dCT duration d dCT is modeled as a single variable that is learned (initialized with 1).


Notice that if we set margin m τ to 0, the second case becomes an L1 loss |x −ŷ|.

we use a small margin m τ to promote some distance between ordered points and prevent con-fusion with equality.


The combination of losses L * shows mixed results, and has lower performance for S-TLM and C-TLM, but better performance for TL2RTL.

it is slowest to compute, and less interpretable, as it is a combined loss.


The GCNs are designed to capture the dependencies between shortcut arcs, while the layer number of GCNs limits the ability to capture local graph information.

in this cases, we find that leveraging local sequential context will help to expand the information flow without increasing the layer number of GCNs, which means LSTMs and GCNs maybe complementary.


The construction of large-scale Knowledge Bases (KBs) like Freebase (Bollacker et al., 2008) and Wikidata (Vrandečić and Krötzsch, 2014) has proven to be useful in many natural language processing (NLP) tasks like question-answering, web search, etc.


(2017) also attempt to mitigate noise in DS through their joint entity typing and relation extraction model.

kBs like Freebase readily provide reliable type information which could be directly utilized.


Traditional approaches to the task of ACE event detection primarily regard multiple events in one sentence as independent ones and recognize them separately by using sentence-level information.

events in one sentence are usually interdependent and sentence-level information is often insufficient to resolve ambiguities for some types of events.


Because of the ambiguity, a traditional approach may mislabel fired in S1 as a trigger of End-Position event.

if we know died triggers a Die event in S1, which is easier to disambiguate, we tend to predict that fired triggers an Attack event.


Some works (Li et al., 2013;Yang and Mitchell, 2016;Liu et al., 2016b) rely on a set of elaborately designed features and complicated natural language processing (NLP) tools to capture event interdependency.

these methods lack generalization, take a large amount of human effort and are prone to error propagation problem.


There have been some feature-based studies (Ji and Grishman, 2008;Liao and Grishman, 2010;Huang and Riloff, 2012) that construct rules to capture document-level information for improving sentence-level ED.

they suffer from two problems: (1) The features they used often need to be manually designed and may involve error propagation from existing NLP tools; (2) Sentence-level and document-level information are integrated by a large number of fixed rules, which is complicated to construct and it will be far from complete.


(2016) exploits a neural-based method to detect multiple events collectively.

they only use the sentence-level information and ne-glect document-level clues, and can only capture the interdependencies between the current event candidate and its former predicted events.


Including them during training distracts other parsing objectives (compare Core + PP with only analyzing Core in §6).

they do permit improvements on precision for PP attachment by 3.30, especially with our proposed joint decoding.


2017extend HMM or dependency model with valence (DMV) (Klein and Manning, 2004) with multinomials that use word (or tag) embeddings in their parameterization.

they do not represent the embeddings as latent variables.


This is partially because automatically parsing from words is difficult even when using unsupervised syntactic categories (Spitkovsky et al., 2011a).

inducing dependencies from words alone represents a more realistic experimental condition since gold POS tags are often unavailable in practice.


Thus, results from related work trained on gold tags are not directly comparable.

to measure how these systems might perform without gold tags, we run three recent state-of-theart systems in our experimental setting: UR-A E-DMV (Tu and Honavar, 2012), Neural E-DMV (Jiang et al., 2016), and CRF Autoencoder (CRFAE) (Cai et al., 2017).


We inspect the two clusters and the overlapping region in Figure 5, it turns out that the nouns in the separated clusters are words that can appear as subjects and, therefore, for which verb agreement is important to model.

the nouns in the overlapping region are typically objects.


Our approach can also be viewed in connection with generative adversarial networks (GANs) (Goodfellow et al., 2014) that is a likelihood-free framework to learn implicit generative models.

it is nontrivial for a gradient-based method like GANs to propagate gradients through discrete structures.


On the one hand, Kuncoro et al., 2017) proposed a top-down transition-based algorithm, which creates a phrase structure tree in the stack by first choosing the non-terminal on the top of the tree, and then considering which should be its child nodes.

to the bottom-up approach, this top-down strategy adds a lookahead guidance to the parsing process, while it loses rich local features from partially-built trees.


Liu and Zhang (2017a) report that the top-down approach is on par with the bottom-up strategy in terms of accuracy and the in-order parser yields the best accuracy to date on the WSJ.

despite being two adequate alternatives to the traditional bottom-up strategy, no further work has been undertaken to improve their performance.


This will serve as the basis for our reduction of constituent parsing to sequence labeling.

to go from theory to practice, we need to overcome two limitations of the theoretical encoding: non-surjectivity and the inability to encode unary branches.


Examples of practical tasks that can be formulated under this framework in natural language processing are PoS tagging, chunking or named-entity recognition, which are in general fast.

to our knowledge, there is no previous work on sequence labeling methods for constituent parsing, as an encoding allowing it was lacking so far.


While their purpose is also different from ours, as they use this mapping to generate training data for a parsing algorithm based on recursive partitioning using realvalued distances, their encoding could also be applied with our sequence labeling approach.

it has the drawback that it only supports binarized trees, and some of its theoretical properties are worse for our goal, as the way to define the inverse of an arbitrary label sequence can be highly ambiguous: for example, a sequence of n−1 equal labels in this encoding can represent any binary tree with n leaves.


We could achieve such a bigram from the unordered source tree .

that realization is not in fact appropriate for French, so that ordered tree would not be a useful training tree for French.


It ensures that each model's expected bigram counts match those in the POS sequences.

these maximum-likelihood estimates might overfit on our finite data, u and B.


The most commonly used framework for active learning is pool-based active learning, where the learner has access to the entire pool of unlabeled data at once, and can iteratively query for examples.

sequential active learning is a framework in which unlabeled examples are presented to the learner in a stream (Lewis and Gale, 1994).


Uncertainty sampling is a common method in pool-based active learning to identify the best example to improve a classifier.

for a given predicate p, we use this to choose the best label query involving that predicate, picking that object o ∈ O A which is closest to the hyperplane of the classifier for p. it is more challenging to narrow down the number of predicates.


MovieQA (Tapaswi et al., 2016) is most similar to our dataset, with both multiple choice questions and timestamp annotation.

their questions and answers are constructed by people posing questions from a provided plot summary, then later aligned to the video clips, which makes most of their questions text oriented.


For example, given an example sentence from TEMPO-TL (e.g., "The cross is seen for the first time then window is first seen in room"), the model does not need to reason about the ordering of "cross seen for the first time" and "window is seen for the first time" because both moments only happen once in the video.

when considering the sentence "The adult hands the little boy a stick then they begin to walk" (from Figure 3), "begin to walk" could refer to multiple video moments.


Rare word representation has recently enjoyed a surge of interest, owing to the crucial role that effective handling of infrequent words can play in accurate semantic understanding.

there is a paucity of reliable benchmarks for evaluation and comparison of these techniques.


Many of these English word similarity datasets have been translated to other languages to create frameworks for multilingual (Leviant and Reichart, 2015) or crosslingual (Camacho-Collados et al., 2017) semantic representation techniques.

these datasets mostly target words that occur frequently in generic texts and, as a result, are not suitable for the evaluation of subword or rare word representation models.


been regarded as the de facto standard evaluation benchmark for subword and rare word representation techniques.

our analysis shows that RW suffers from multiple issues: (1) skewed distribution of the scores, (2) low-quality and inconsistent scores, and as a consequence, (3) low inter-annotator agreement.


Knowledge-based methods (Lesk, 1986;Moro et al., 2014;Basile et al., 2014) exploit the lexical knowledge like gloss to infer the correct senses of ambiguous words in the context.

supervised feature-based methods (Zhi and Ng, 2010;Iacobacci et al., 2016) and neural-based methods (Kågebäck and Salomonsson, 2016;Raganato et al., 2017a) usually use labeled data to train one or more classifiers.


As shown in Table 1, the local word "football" is crucial for distinguishing the sense of word "play".

in more complex sentences such as "Investors played it carefully for maximum advantage" 1 , sentence-level information is necessary.


We use the validation set (SE7) to find the optimal hyper parameters of our models: the word embedding size d w , the hidden state size d s of LSTM, the optimizer, etc.

since there are no adverbs and adjectives in SE7, we randomly sample some adverbs and adjectives from training dataset into SE7 for validation.


While the basic senses of tight-e.g., being physically close together or firmly attached-conflict with the more abstract budget, the metaphoric use as meaning limited can be readily understood.

the use of penumbra in (2) is more creative and novel.


It is only recently that any work has introduced larger-scale novel metaphor annotations (Parde and Nielsen, 2018).

to our approach, they collect annotations on a relation level (see also Section 2).


On the one hand, this is a sensible approach because generally the context of a word determines its metaphoricity (and indeed, its novelty in case of metaphoric use).

such annotations lack the flexibility and ease of use of tokenbased annotations for which the context is not defined a priori.


For example, in this way the metaphor "[...] the artistic temperament which kept her tight-coiled as a spring [...]" (0.514) is treated as novel, while "To quench [thirst] is more than to refresh [...]" (0.424) is treated as conventionalized.

since we provide the scores, this threshold can be adjusted to suit a given application.


Another artifact of using Wikipedia as the background corpus can be seen on the left: try is only annotated as metaphoric in infinitive-compounds that are decidedly conventionalized (e.g., "trying to look"), yet it appears comparatively seldom in Wikipedia.

we chose to use Wikipedia instead of the BNC in order to have an out-ofdomain comparison with a more contemporary, larger background corpus.


Third, they update the memory according to whether the returned value is strictly the same as the target.

synonyms are common in natural language text.


Similarly to MPSG, AdaGram introduces latent variables for word sense indexes in the input text.

unlike MPSG, AdaGram does not assume a fixed number of word senses.


2) disambiguated skip-gram adopts the softmax model used in MPSG.

in contrast to the previous works, disambiguated skip-gram learns a parametric model for the conditional probability distribution over senses of the center word given the context words (Eq.


This does not mean, however, that the number of senses learned by Ada-Gram is independent of model hyper-parameters.

the number of senses learned by AdaGram is directly controlled by the hyper parameter α in the Dirichlet process used to define the prior over word meanings (Bartunov et al., 2016).


We choose to optimize this objective with a biased but low-variance gradient estimator.

parallel to this work there has been a significant progress in gradientbased training of models with discrete latent variables.


Training PIXIE amounts to inferring the posterior over the visual latent factors, p θ (z|x, y), as well as finding the decoder's parameters θ, the word and context embeddings, e and v, that maximize the likelihood (6).

as in many complex probabilistic models, the likelihood (due to the integral over z) and the posterior are intractable.


learn two separate functions f a and f b ) to predict Y a and Y b in a single-task learning setup.

if T a and T b are related somehow, either explicitly or implicitly, TL and MTL can improve the generalization of either task or both (Caruana, 1997;Pan and Yang, 2010;Mou et al., 2016).


Word embeddings such as Word2vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014) have been widely recognized for their ability to capture linguistic regularities (including syntactic and semantic relations).

no linguistic property of their prepositional embeddings is known; to the best of our knowledge, we propose the first sense-specific prepositional embeddings and demonstrate their linguistic regularities.


Our model unifies these two approaches into an autoencoder.

we have a different goal: that of creating or improving word representations.


We emphasize that our method trains exclusively on the definitions and is thus applicable to any electronic dictionary.

in order to evaluate the quality of em-beddings on unseen definitions, WordNet relations comes in handy: we use the sets of synonyms to split the dictionary into a train set and a test set, as explained in Section 7.


Both dict2vec and retrofitting improve with regards to word2vec on similarity benchmarks and seem roughly on par.

dict2vec fails to improve on relatedness benchmarks, whereas retrofitting sometimes improves (as in RG, MEN, and MT), sometimes equals (SCWS) and does worse (353).


These widely-used representations often significantly improve performance in downstream tasks, as they are able to leverage large amounts of unstructured data.

most of the popular collections of word embeddings assign only one vector to each word, thus shifting the burden of word disambiguation to deeper, task-specific layers, which commonly rely on data of much smaller scales.


Specifically, we make following contributions: • We build a dataset consisting of 11.3k sentences containing a frequently used comparator "oe" for simile recognition in Chinese, which can support data-driven approaches.

to English, datasets on simile or  • We propose a neural multitask learning framework jointly optimizing three tasks: simile sentence classification, simile component extraction and language modeling.


If the component extractor knows that a sentence contains a simile, it would be more confident to extract the tenor and the vehicle.

if the component extractor tells the classifier that the tenor and the vehicle likely exist, the classifier gets additional information for decision.


For each word, instead of using hidden state h t only, we combine h t and its score vector p t as a representation s t : Since p t is directly related to the component extraction task, this labeling connection operation increases the interaction between the two tasks.

words in a sentence should not contribute the same for classification.


As shown in Table 4, the optimized pipeline performs better than the strongest multitask learning setting.

in all settings, the precision scores are lower compared with the recall scores.


Finally, and most similar to our approach, several models have been proposed that directly compare subtrees between two sentences (Chen et al., 2017;Zhao et al., 2016).

all of these models are pipelined; they obtain the sentence structure in a non-differentiable preprocessing step, losing the benefits of end-to-end training.


This also validates the observation of (Chen et al., 2017b), which shows what the sub vector in the ESIM model is looking out for contradictory information.

our architecture allows the inspection of these vectors since they are compressed via factorization, leading to larger extents of explainability -a quality that neural models inherently lack.


State of the art models using deep neural networks have become very good in learning an accurate mapping from inputs to outputs.

they still lack generalization capabilities in conditions that differ from the ones encountered during training.


in case of the fastText [MIMIC-III] embeddings knowledge-directed attention was beneficial to both models, as shown in Table 7.


Recently, neural network based approaches (Dong and Lapata, 2016;Jia and Liang, 2016;Xiao et al., 2016;Guu et al., 2017;Dong et al., 2018) have achieved promising performance in semantic parsing.

neural network approaches are data hungry, which performances closely correlate with the volume of training data.


Specifically, the optimal parameters θ * are obtained as follow: Gradient descent is used to search out optimal parameters θ * .

mLE fails to consider the fact that semantically equivalent regular expressions might be syntactically different.


We encourage the model to generate regular expressions to maximize expected rewards instead of MLE.

concretely, we train the model parameters θ to maximize the following objective function: to compute the expected reward, we need to go over all possible regular expressions, and the number of all possible regular expressions is infinite.


A challenge here is that we do not know the samples before performing the policy-gradient method.

we find that there is a high chance to get the same samples repeatedly when the model is pre-trained using MLE on the training set, because sampling is following the distribution learned by the pre-trained model.


SENNA and other typical word embeddings always assign an identical vector to each word regardless of the input context.

eLMo assigns different vectors to each word depending on the input context.


Similar to (Rabinovich et al., 2017), our model structures the decoder as a collection of recursive modules.

as we discussed in the related work section, we make use of a SQL specific grammar to guide the decoding process, which allows us to take advantage of SQL queries' welldefined structure.


The Seq2Seq models suffer from generating ungrammatical queries, yielding very low exact matching accuracy on Hard and Extra Hard SQL queries.

our model generates valid SQL queries by enforcing the syntax.


Evaluating such forms is crucial to the development of parsing algorithms.

there is no method directly available for evaluation.


cal form does not need to be given by the annotator (QA denotations, for example, require the annotator to know the correct answer to the question -an assumption which doesn't hold for end-users who asked the question with the goal of obtaining the answer).

our goal is to leverage natural language feedback and corrections that may occur naturally as part of the continuous interaction with the non-expert end-user, as training signal to learn a semantic parser.


For at least one of the conditions in the SQL query, its column name is not explicitly mentioned in the question, not even in paraphrased form.

there are still partial semantic clues for inference.


generates each of these individual components by first predicting COLUMN and OP, and then generating VALUE using pointer networks (Vinyals et al.).

we propose a two step approach to tackle this problem in the reverse way, taking advantage of the content of table t as additional knowledge: (i) Generate the condition VALUE from the question, and (ii) predict which COLUMN and OP apply to this VALUE.


BASELINE refers to a baseline for our models where the WHERE clause accuracy is computed by assuming that each candidate (COLUMN, VALUE) pair is included with corresponding OP being equality.

uPPERBOuND accuracy is computed by assuming f cond and f op makes 100% correct mapping of whether to include a candidate (COLuMN, VALuE) pair and which OP to apply on this condition.


study how to answer user questions with table cells from millions of HTML tables.


It yields the lowest crossing arcs rate since lots of concepts are not aligned.

empty Alignment Issue: our hybrid aligner yields less empty alignments.


The underlying hypothesis is that enriching word vector representations with polysemic information should express itself in performance gains in these tasks.

this hypothesis has never been tested directly, and the ability of word similarity tasks to directly benefit from polysemic information must first be validated if they are to serve as genuine evaluation sets in polysemy research.


very, totally) strengthens the adjectives it modifies.


When using the linear kernel or cosine similarity, the estimator of PHSIC in feature space (14) is as follows: Generally in kernel methods, a feature map φ(•) induced by a kernel k(•, •) is unknown or highdimensional and it is difficult to compute estimated values in feature space 6 .

when we use the linear kernel or cosine similarity, feature maps can be explicitly determined (Equation 19).


First, information-theoretic MI (Cover and Thomas, 2006) and its variants (Suzuki et al., 2009;Reshef et al., 2011) are the most commonly used dependence measures.

to the best of our knowledge, there is no practical method of computing MIs for large-multi class high-dimensional (having a complex generative model) discrete data, such as sparse linguistic data.


To address the summarization alignment, former studies try to apply an attention mechanism to measure the saliency/novelty of each candidate word/sentence (Tan et al., 2017), with the aim of locating the most representative content to retain primary coverage.

toward summarizing a related work section, authors should be more creative when organizing text streams from the reference collection, where the selected content ought to highlight the topic bias of current work, rather than retell each reference in a compressed but balanced fashion.


It needs to remove the unnecessary information and select salient information from the input document to produce a condensed summary.

it is difficult for the basic encoderdecoder framework to learn the process of salient information selection, which has also been noticed by several previous work (Tan et al., 2017a,b).


ive text summarization aims to shorten long text documents into a human readable form that contains the most important facts from the original document.

the level of actual abstraction as measured by novel phrases that do not appear in the source document remains low in existing approaches.


(2016); Wang and Jiang (2017) explore a compare and aggregate framework to directly capture the wordby-word matching between two paired sentences.

these approaches suffer from the problem of high matching complexity, since a similarity matrix between pairwise words needs to be computed, and thus it is computationally inefficient or even prohibitive when applied to long sentences (Mou et al., 2016).


The adaptive context-aware filter generation mechanism proposed here bears close resemblance to attention mechanism (Yin et al., 2016;Bahdanau et al., 2015;Xiong et al., 2017) widely adopted in the NLP community, in the sense that both methods intend to incorporate rich contextual information into text representations.

attention is typically operated on top of the hidden units preprocessed by CNN or LSTM layers, and assigns different weights to each unit according to a context vector.


We attribute the improvement to two potential advantages of our AdaQA model: (i) for the two previous baseline methods, the interaction between question and answer takes place either before or after convolution.

in our AdaQA model, the communication between two sentences is inherent in the convolution operation, and thus can provide the abstracted features with more flexibility; (ii) the bidirectional filter generation mechanism in our AdaQA model generates co-dependent representations for the question and candidate answer, which could enable the model to recover from initial local maxima corresponding to incorrect predictions (Xiong et al., 2017).


The typical translation process is performed with either off-the-shelf machine translation (MT) systems or multilingual dictionaries (Nie, 2010).

mT based approaches are far from being a suitable solution for solving CLIR problems (refer to detailed analysis in (Zhou et al., 2012)).


A recent work (Gupta et al., 2017) tries to learn taskspecific representation for CLIR.

their model only captures ranking signals in monolingual settings, which does not necessarily generalize well in CLIR.


A recent work (Gupta et al., 2017) tries to learn task-specific embeddings for CLIR.

it learns ranking signals by preserving pairwise ranking in monolingual settings prior to a transfer learning process to another language, which does not necessarily generalize well in CLIR.


OT is a general mathematical toolbox used to evaluate correspondence-based distances and establish mappings between probability distributions, including discrete distributions such as point-sets.

the nature of mono-lingual word embeddings renders the classic formulation of OT inapplicable to our setting.


Furthermore, both Russian and Spanish are prodrop languages (Haspelmath, 2001) and share syntactic phenomena, such as dative subjects (Moore and Perlmutter, 2000;Melis et al., 2013) and differential object marking (Bossong, 1991), which might explain why ES is closest to RU overall.

english appears remarkably isolated from all languages, equally distant from its germanic (De) and romance (FR) cousins.


(2017b) also leverage optimal transport distances for the cross-lingual embedding task.

to address the issue of nonalignment of embedding spaces, their approach follows the joint optimization of the transportation and procrustes problem as outlined in Section 2.2.


In principle, one simply looks beyond single sentences for co-occurring entity pairs.

this can introduce many false positives and prior work used a small sliding window and filtering (minimal-span) to mitigate training noise.


The implicit assumption behind this method is that machine-generated attention should mimic human rationales.

rationales on their own are not adequate substitutes for machine attention.


Existing methods commonly adapt the classifier by aligning the representations between the source and target domains (Glorot et al., 2011;Chen et al., 2012;Zhou et al., 2016;Ganin et al., 2016;Zhang et al., 2017).

our model adapts the mapping from rationales to attention; thus after training, it can be applied to different target tasks.


1 Deep learning models work best when trained on large amounts of labeled data.

acquiring labels is costly, motivating the need for effective semi-supervised learning techniques that leverage unlabeled examples.


This is similar to CVT in that it exposes the model to a restricted view of the input.


The availability of large scale annotated corpora for coreference is essential to the development of the field.

creating resources at the required scale via expert annotation would be too expensive.


These include medium-scale multilingual datasets such as ONTONOTES (Pradhan et al., 2007;Weischedel et al., 2011), which led to the most recent evaluation campaigns, in particular CONLL (Pradhan et al., 2012, and are used in most current research (Björkelund and Kuhn, 2014;Martschat and Strube, 2015;Clark and Manning, 2016;Lee et al., 2017).

there are still many languages and domains for which no such resources are available, and even for English much larger corpora than ONTONOTES will eventually be required.


However, there are still many languages and domains for which no such resources are available, and even for English much larger corpora than ONTONOTES will eventually be required.

annotating data on the scale required to train state of the art systems using traditional expert annotation would be unaffordable.


1 A second coreference corpus created using crowdsourcing (in the context of a trivia game) also exists, the Quiz Bowl dataset (Guha et al., 2015).

2 such existing corpora are not widely used yet.


For this substask, most previous work (Poesio et al., 2004;Lassalle and Denis, 2011;Hou et al., 2013b) calculate semantic relatedness between an anaphor and its antecedent based on word co-occurrence counts using certain syntactic patterns.

such patterns only consider head noun knowledge and hence are not sufficient for bridging relations which require the semantics of modification.


Hou (2018) created word embeddings for bridging (embeddings PP) by exploring the syntactic structure of noun phrases (NPs) to derive contexts for nouns in the GloVe model.

embeddings PP only contains the word representations for nouns.


For embeddings PP, the result on using NP head + modifiers (31.67%) is worse than the result on using NP head (33.03%).

if we apply embeddings PP to a bridging anaphor's head and modifiers, and only apply embeddings PP to the head noun of an antecedent candidate, we get an accuracy of 34.53%.


Another new corpus for bridging is the second release of the ARRAU corpus, which contains 5,512 bridging pairs in three different domains (Poesio et al., 2018).

most bridging links in ARRAU are purely lexical bridging pairs, and only a small subset of the annotated pairs contains truly anaphoric bridging anaphors .


In parallel, more general work on common sense reasoning aims to develop a repository of common knowledge using semi-automatic methods (e.g., Cyc (Lenat, 1995) and ConceptNet (Liu and Singh, 2004)).

such knowledge bases are necessarily incomplete.


The man couldn't lift his son because he was so weak.

answer: the man (agent) Evidence and labels: "I was so weak that I couldn't lift" → Ea (query terms in bold) "She was so weak she couldn't lift" → Ea "I could not stand without falling immediately and I was so weak that I couldn't lift" → Ea "It hurts to lift my leg and its kind of weak" → EP Stats and resolution: agent evidence strength: 97 Patient evidence strength: 72 Number of scraped sentences: 109 Resolution: agent Table 5: Example Resolution for a WSC problem.


As a consequence, two vectors of the resulting space will be close if their corresponding nodes occur in topological proximity within the graph.

while such a topology allows perhaps for meaningful comparisons between points in this space, it is not directly compatible with the task of mapping text to entities.


The textual descriptions are treated as short documents, and each word in them is assigned a specific TF-IDF value, forming the set of textual features for the specific entity.

the KB graph is extended in the following way: Let t c be the set of textual features for an entity c; then, for each t in t c , we add an edge (c, t) with weight tf-idf c (t), where tf-idf c (t) is the tF-IDF value of t with respect to c. to Perozzi et al.


The purpose of the classification task ( §4.3) is to provide a direct comparison of the textually enhanced vectors against vectors produced by the original graph, but independently of the compositional part.

the text mapping experiments ( § §4.1, 4.2) evaluate the overall architecture of Fig.


More recently, several models (Shi and Weninger, 2017;Xie et al., 2016) have been proposed to handle unseen entities by leveraging text descriptions.

to these approaches, our model deals with long-tail or newly added relations and focuses on one-shot relational learning without any external information, such as text descriptions of entities or relations.


When evaluating existing embedding models, during training, we use not only the triples of background relations but also all the triples of the training relations and the one-shot training triple of those validation/test relations.

since the proposed metric model does not require the embeddings of query relations, we only include the triples of the background relations for embedding training.


These obstacles have motivated the work on automatic inference of REs (Banko et al., 2007;Li et al., 2008;Bartoli et al., 2018) where the objective it to develop approaches that are fast and deployable in real time.

the existing approaches tend to require large number of examples to cover both the alphabet and the possible syntactic patterns.


Recently, a KG embedding method which utilizes temporal scopes was proposed in (Jiang et al., 2016).

instead of directly incorporating time in the learned embeddings, the method proposed in (Jiang et al., 2016) first learns temporal order among relations (e.g., wasBorIn → wonPrize → diedIn).


We hypothesize that this phenomenon emerges due to TDNS (Equation 2).

in case of link prediction, we notice that the extra samples are affecting performance as they originate from the KG itself.


Recent research efforts have shown that neural architectures can be effective in conventional information extraction tasks such as named entity recognition, yielding state-of-the-art results on standard newswire datasets.

despite significant resources required for training such models, the performance of a model trained on one domain typically degrades dramatically when applied to a different domain, yet extracting entities from new emerging domains such as social media can be of significant interest.


(2012) show that domainspecific word embeddings tend to perform better when used in supervised learning tasks.

1 maintaining such an improvement in the transfer learning process is very challenging.


We designed this experimental setup based on the following considerations: • Challenging: Newswire is a well-studied domain for NER and existing neural models perform very well (around 90.0 F1-score (Ma and Hovy, 2016)).

the performance drop dramatically in social media data (around 60.0 F-score (Strauss et al., 2016)).


Thus, the transferred information is diluted while we train the target model with more data.

our transfer method explicitly saves the transferred information in the base part of our target model, and we use separate learning rates to help the target model to preserve the transferred knowledge.


(Shen et al., 2017) proposed to link entity mentions to an HIN such as DBLP and IMDB.

their articles are collected from the Internet through searching and thus are not related to the target entities.


One of the first such attempt was made by (Wang, 2009) on 311 clinical notes from an Intensive Care Unit (ICU) department of the single hospital -Royal Prince Alfred Hospital (RPAH).

no specific guidelines on how that data were annotated are available.


Typically, NER models are built upon conditional random fields (CRF) with the IOB or IOBES tagging scheme Ma and Hovy, 2016;Lample et al., 2016;Ratinov and Roth, 2009;Finkel et al., 2005).

such design cannot deal with multi-label tokens.


More interestingly, compared to multi-word entity mentions, matched unigram entity mentions are more likely to be false-positive labels.

such false-positive labels will not introduce incorrect labels with the Tie or Break scheme, since either the unigram is a true entity mention or a false positive, it always brings two Break labels around.


(2016a) use a cross-lingual WIKIFIER to facilitate cross-lingual NER.

they do not explicitly address the case where the target entity does not exist in Wikipedia.


Most work in machine reading focuses on question answering problems where the answer is directly expressed in the text to read.

many real-world question answering problems require the reading of text not because it contains the literal answer, but because it contains a recipe to derive an answer together with the reader's background knowledge.


We generally assume that the question is underspecified, in the sense that the question often does not provide enough information to be answered directly.

an agent can use the supporting rule text to infer what needs to be asked in order to determine the final answer.


Complex Question Answering, in which the question corresponds to multiple triples in knowledge base, has attracted researchers' attentions recently (Bao et al., 2014;Xu et al., 2016;Berant and Liang, 2014;Bast and Haussmann, 2015;Yih et al., 2015).

most of existing solutions employ the predefined patterns or templates in understanding complex questions.


For example, the correct SQG of question "In which countries do people speak Japanese" has only two nodes "countries", "Japanese".

we recognized the phrase "people" as a variable node by mistake and connected it to both "countries" and "Japanese" to generate a wrong SQG Q S .


Character sequence would give information which helps to relieve the OOV problem, as many English words share the same stem and differ only in prefix or suffix.

this is not the case in Chinese, and we observe no significant improvement incorporating character-level embedding into our system.


Note that this formulation is in similar spirit to highway networks (Srivastava et al., 2015).

since our gating function is learned via reasoning over multi-granular sequence blocks, it captures more compositionality and long range context.


Unless stated otherwise, the encoder in the pointer layer for span prediction models also uses DCU.

for the Hybrid DCU-LSTM models, answer pointer layers use BiLSTMs.


When compared with a BiL-STM of equal output dimensions (150d), we find that our DCU model performs competitively, with less than 1% deprovement across all metrics.

the time cost required is significantly reduced.


Here, we note that Sim-DCU does not produce reasonable results at all, which seems to be in similar vein to results on SearchQA, i.e., a recursive cell that processes word-by-word is mandatory for span prediction.

our results show that it is not necessary to construct gates in a word-by-word fashion.


Our work is most similar to N2NMN (Hu et al., 2017) model, which learns both semantic operators and the layout in which to compose them.

optimizing the layouts requires reinforcement learning, which is challenging due to the high variance of policy gradients, whereas our chart-based approach is end-to-end differentiable.


As a result, a large percentage of CoQA answers are named entities or short noun phrases, much like those in SQuAD.

the asymmetric nature of forces students to ask more exploratory questions whose answers can be potentially be followed up on.


Dependency paths focus on hidden features at syntactic and functional perspective, which is a good complementary to sentential encoding results.

performances drop by 2.17 if only dependency information is used, we find that under certain dependency structures, crucial words (bolded) are not in the path between the answer and the focus mention (underlined), for example, "who did draco malloy end up marrying" and "who did the philippines gain independence from".