DRL-Sense: Deep Reinforcement Learning for Multi-Sense Word Representations
This paper proposes DRL-Sense—a multisense word representation learning model, to address the word sense ambiguity issue, where a sense selection module and a sense representation module are jointly learned in a reinforcement learning fashion. A novel reward passing procedure is proposed to enable joint training on the selection and representation modules. The modular design implements pure senselevel representation learning with linear time sense selection (decoding). We further develop a non-parametric learning algorithm and a sense exploration mechanism for better flexibility and robustness. The experiments on benchmark data show that the proposed approach achieves the state-of-the-art performance on contextual word similarities and comparable performance with Google’s word2vec while using much less training data.
1 Introduction
Recently, deep learning methodologies have dominated several research areas in natural language processing (NLP), such as machine translation, language understanding, and dialogue systems. However, most of applications usually utilize word-level embeddings to obtain semantics. Considering that natural language is highly ambiguous, the standard word embeddings may suffer from polysemy issues. Neelakantan et al. (2014) pointed out that, due to triangle inequality in vector space, if one word embedding has two different senses, the sum of the distance between the word embedding and its synonymous embedding in each sense would be larger or equal to the distance between these synonymous embed-
dings1. Because this inaccurate distance measurement may degrade downstream NLP tasks, multisense word representations are proposed to address the ambiguity issue in a single word embedding scenario (Reisinger and Mooney, 2010; Huang et al., 2012)
This paper proposes DRL-Sense—a novel reinforcement learning based framework for learning multi-sense representations, which is composed by two main modules: a sense selection module for inferring the most probable sense for a word given its context, and a sense representation module for representing word senses in a continuous space. Our modular design implements pure sense-level representation learning while maintaining linear time sense selection. The proposed model is optimized by reinforcement learning and incorporates a non-parametric algorithms and a sense exploration mechanism without changing the network architecture2. Our contributions are four-fold: • We are among the first to study reinforcement
learning for sense embedding learning, taking account of the Markov property in sense selection given local contexts. • DRL-Sense is the first system that achieves
pure sense-level representation learning with linear time complexity on sense selection. • We develop non-parametric learning and
sense exploration mechanisms for a general neural sense selection module to achieve better flexibility and robustness. • Our experimental results show the state-of-
the-art performance on contextual word similarities and comparable performance with word2vec (Mikolov et al., 2013b) using only 1/100 size of training data.
1d(bank, finance) + d(bank, coast) ≥ d(finance, coast) 2The code will be released after the paper gets accepted.
2
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
2 Related Work
There are three dominant types of approaches for learning multi-sense word representations in the literature: 1) clustering methods, 2) probabilistic modeling methods, and 3) lexical ontology based methods. Our reinforcement learning based approach can be loosely connected to clustering methods and probabilistic modeling methods.
Reisinger and Mooney (2010) is the first work to propose multi-sense word representations on vector space based on clustering techniques. With the power of deep learning, some work exploit neural networks to learn embeddings with senseannotated corpus based on clustering (Huang et al., 2012; Neelakantan et al., 2014). Chen et al. (2014) replaced the clustering procedure with a word sense disambiguation model based on WordNet (Miller, 1995). Kågebäck et al. (2015) further exploited a weighting mechanism on context in the clustering procedure and evaluated their system on word sense induction. Vu and Parker (2016) proposed an iterative process on the twostage clustering-embedding learning framework. Moreover, Guo et al. (2014) leveraged bilingual resources to perform clustering. However, most of the above approaches separates the clustering procedure and the representation learning procedure, which may suffer from the error propagation issue. Instead, our reinforcement learning model utilizes a reward signal to propagate the statistical information from sense representations to optimize sense selection.
In order to make the model more flexible, probabilistic modeling methods are usually utilized for learning multi-sense emnbeddings, where Tian et al. (2014) and Jauhar et al. (2015) conducted probabilistic modeling with EM training. Li and Jurafsky (2015) exploited the Chinese Restaurant Process and demonstrates efficacy of multi-sense word representations on several downstream NLP tasks. Furthermore, Bartunov et al. (2016) developed a non-parametric Bayesian extension on the skip-gram model (Mikolov et al., 2013b). However, all above methods fail to maintain pure sense-level representation learning due to the complicated computation in their EM algorithms (Bartunov et al., 2016), where sense-level embeddings are trained with word-level embeddings.
Recently, Qiu et al. (2016) proposed an EM algorithm to learn representations without wordlevel embeddings, where the computational cost
is high when selecting the sense identity sequence because the time for searching all sense combination within a context window is exponential. In contrast, by exploiting embeddings in the sense selection module, our model performs linear time sense selection, while maintaining a pure senselevel representation learning module. Another difference between the prior work and ours is that, Qiu et al. (2016) used WordNet (Miller, 1995) to obtain the number of senses for words, and our solution supports non-parametric learning for automatically deciding the sense number of each word.
Furthermore, most related work either directly exploited sense representations (Neelakantan et al., 2014; Qiu et al., 2016) or combined with online expectation sense counts (Li and Jurafsky, 2015; Jauhar et al., 2015) to select senses. The proposed approach incorporates a sense selection module, which can be learned together with the representation learning module through reinforcement learning.
Unlike a lot of relevant work that requires additional resources such as the lexical ontology (Rothe and Schütze, 2015; Jauhar et al., 2015; Chen et al., 2015; Iacobacci et al., 2015) or bilingual data (Guo et al., 2014; Ettinger et al., 2016; Šuster et al., 2016), which may not be available in the target task, our model can be trained on the corpus only. Also, some prior work proposed to learn topical embeddings and word embeddings jointly in order to consider the contexts (Liu et al., 2015a,b), whereas this paper focuses on learning multi-sense word embeddings.
3 Proposed Approach: DRL-Sense
This work proposes a framework to learn two key modules for multi-sense word representations: a sense selection module and a sense representation module. The sense selection module decides which sense to use given a text context, whereas the sense representation module learns meaningful representation for sense by their statistical characteristics.
Considering that the sense representation module requires the sense identity of each word from the sense selection module, and the sense selection module may also benefit from the semantics carried by sense representations, these two modules should be tangled. Hence, a naive two-stage algorithm or two separate learning algorithms proposed by prior work are not optimal. We propose
3
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
𝐶𝑡−𝑚 𝐶𝑡+𝑚𝐶𝑡 = 𝑤𝑖… …
𝑞( ഥ𝐶𝑡 , 𝑧𝑖1) 𝑞( ഥ𝐶𝑡, 𝑧𝑖2) 𝑞( ഥ𝐶𝑡, 𝑧𝑖3)
𝐶𝑡′−𝑚 𝐶𝑡′+𝑚𝐶𝑡′ = 𝑤𝑗… …
𝑞(𝐶𝑡′, 𝑧𝑗1) 𝑞(𝐶𝑡′, 𝑧𝑗2) 𝑞(𝐶𝑡′, 𝑧𝑗3)
𝑧𝑖1
Sense Representation Module
…𝑃(𝑧𝑗2|𝑧𝑖1) 𝑃(𝑧𝑢𝑣|𝑧𝑖1)
→ Q-learning reward signal
← greedy sense selection
Sense Selection Module
← Q -learn in g rew ard sign al → greed y sen se selectio n
Sense selection for target word 𝐶𝑡 Sense selection for collocated word 𝐶𝑡′
negative sampling
matrix 𝑄𝑖
matrix 𝑉
matrix 𝑈 matrix 𝑃
matrix 𝑄𝑗
matrix 𝑃
sample collocation Sense Selection Module
Figure 1: The RRL-Sense architecture illustration. Two shared-weight sense selection modules surround a sense representation module. The sense selection module infers the target word sense given its context. The selected senses are passed to the sense representation module for learning sense embeddings. Then, the sense embedding collocation is passed back to the sense selection modules for reinforcement learning.
to learn two modules in a reinforcement learning manner by introducing a novel reward passing procedure to enable joint training. In addition, a nonparametric learning algorithm is incorporated to enable automatic sense induction, where the number of senses for each word is not predefined but automatically learned for practical usage. Also, simple mechanisms are applied to allow the sense exploration while incorporating the sense selection prior.
3.1 Model Architecture
Our model architecture is illustrated in Figure 1 and detailed below.
3.1.1 Sense Selection Module
The sense selection module decides the sense for a word given its context. Assuming that a word sense is determined by the local context, we use the Markov property to formulate the sense selection module as a Markov Decision Process (MDP) to infer the most probable sense based on its local context. Given a corpus C and a context window m, we extract a local context C̄t = {Ct−m, ..., Ct−1, Ct+1, ..., Ct+m} as a state St, and the selection of a sense zik ∈ Zi for a word wi = Ct given context C̄t as an action Aik given the state St, where Zi is the set of senses of word wi. However, this formulation lacks a central element of MDP: a reward signal for measuring the fitness of the selected sense zik. Therefore, if we have meaningful sense representations containing statistical estimation, we can use the estimation as a surrogate reward to simulate the reward signal in the sense selection module.
3.1.2 Sense Representation Module
A successful sense selection module can be applied to each word in a corpus for obtaining its sense identity. Various techniques about word embeddings can be directly employed after mapping all words in a corpus to its sense identity. The typical method is to formulate the sense selection problem as a maximum likelihood estimation (MLE) problem for the collocation likelihood. In this paper, we use the skip-gram formulation (Mikolov et al., 2013b), because it is more light-weight and thus requires less training time: only two sense selections are required for stochastic training, while GloVe (Pennington et al., 2014) and continuous bag-of-words (CBOW) (Mikolov et al., 2013a) require the whole corpus and the whole context window for sense selection respectively.
Specifically, we first create input sense embedding matrixU and collocation estimation matrix V as the representations to be learned. Given a word wi and collocated word wj with corresponding local contexts, we map them to their sense identities as zik and zjl by the sense selection module, and maximize the sense collocation log likelihood logCo(·) using a negative sampling approximation (Mikolov et al., 2013b),
logCo(zik, zjl | U, V ) = log σ(UTzikVzjl) (1)
+ M∑ v=1 Ezuv∼pneg(z)[log σ(−U T zik Vzuv)],
where pneg(z) is the distribution over all senses for negative samples.
4
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
3.2 Reinforcement Learning
With the above two modules, we treat the sense collocation likelihood as a surrogate reward signal for the sense selection module, and then cast the whole formulation as a reinforcement learning task: An optimal policy πθ with parameters θ should select the sense that leads to an optimal collocation likelihood in (1). In order to estimate the sense collocation likelihood, two senses, zik and zjl, should be determined, which leads a two-step MDP: one for the target word wi and the other for the collocated word wj within a context window. A policy gradient algorithm (Sutton et al., 1999) can formulate the policies as decision probability distributions πiθ(zik|C̄t) and π j θ(zjl|C̄t′) for words wi, wj given their contexts C̄t, C̄t′ . Hence, a joint formulation can be written as follows,
max U,V,θ Ek∼πiθ(·)[El∼πjθ(·)[logCo(zik, zjl | U, V )]]. (2) The neat formulation is differentiable and supports stochastic optimization using doubly stochastic gradient (Lei et al., 2016). However, there are two major drawbacks of this formulation. First, it requires a policy πθ to fit a valid probability distribution, and this constraint underestimates the probability of senses that have not been selected yet during optimization. Second, if using stochastic gradient ascent (detailed in Appendix A) to optimize equation (2), it would actually lower the probability estimation for the selected sense because logCo(·) ≤ 0, which is not desirable if the model accurately select the right sense.
3.2.1 Q-Learning Formulation
To address the above issue, a Q-learning algorithm (Mnih et al., 2013) with a novel reward passing procedure is applied to this task, because it optimizes the selected action (sense) independently to the other actions. Also, considering that the Qlearning algorithm is not constrained by a jointprobabilistic formulation, we reduce the range of the reward function to facilitate the training process by replacing the collocation log likelihood logCo ∈ (− inf, 0] with (its monotonous) likelihood Co ∈ [0, 1] as the reward function (Mnih et al., 2013).
Our model utilizes a CBOW architecture to model the fitness for a sense zik of a wordwi under a local context C̄t. Specifically, given a word embedding matrix P , the local context can be modeled as the summation of word embeddings from
Algorithm 1: DRL-Sense Training Algorithm for wi = Ct ∈ C do
sample wj = Ct′(0 < |t′ − t| ≤ m); k = arg maxk′ q(C̄t, zik′); l = arg maxl′ q(C̄t′ , zjl′); optimize U, V by (1) for the sense representation module; optimize P,Q by (4) for the sense selection module;
its context C̄t. The fitness representation of the selected sense can be formulated with a 3-mode tensorQ, whose dimensions denote words, senses, and latent variables. Then we model the Q-value, q(C̄t, zik), for selecting the sense zik under the context C̄t as
q(C̄t, zik) = σ(Q T ik ∑ j∈C̄t Pj), (3)
where σ is a sigmoid function to form a valid probabilistic estimation.
3.2.2 DRL-Sense Training
To jointly train sense selection and sense representation modules, we first sample a collocated word pair wi, wj with respective contexts C̄t, C̄t′ , and use the estimated Q-value to select the most probable senses zik, zjl. The selected senses are passed to the sense representation module to optimize the sense collocation likelihood. Afterwards, the estimated collocation likelihood is passed back as a reward signal to optimize the sense selection module. The loss function is defined as cross-entropy H(·) due to the probability distribution.
min P,Q
H(Co(zik, zjl | U, V ), q(C̄t, zik))
+H(Co(zik, zjl | U, V ), q(C̄t′ , zjl)). (4)
Algorithm 1 shows the proposed DRL-Sense model training procedure. There are two major contributions in our modular design. First, efficient sense selection with word embeddings and pure sense representation learning are simultaneously achieved. Second, reinforcement learning allows both modules to be jointly trained.
3.3 Non-Parametric Learning
In order to automatically discover the sense number for each word, we incorporate word sense induction into our model (Song et al., 2016). Instead
5
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
𝐶𝑡−𝑚 𝐶𝑡+𝑚
𝑞( ഥ𝐶𝑡 , 𝑧𝑖1) 0.5 0.5 𝑞( ഥ𝐶𝑡 , 𝑧𝑖2)
If 𝑞( ഥ𝐶𝑡, 𝑧𝑖1) < 0.5 If … < 0.5
weight vector zero vector selected vector
𝐶𝑡 = 𝑤𝑖… … 𝐶𝑡−𝑚 𝐶𝑡+𝑚
𝑞( ഥ𝐶𝑡 , 𝑧𝑖1) 0.5
𝐶𝑡 = 𝑤𝑖… …
Sense Selection Module
Figure 2: The proposed non-parametric algorithm with initialization. The red block indicates the newly discovered sense zi2 for the target word wi based on the algorithm.
of non-parametric clustering (Neelakantan et al., 2014) or the stochastic process (Li and Jurafsky, 2015) applied by prior work, this paper proposes a novel mechanism in a general neural model to conduct non-parametric sense discovery.
Because the formulated Q-value is a probabilistic estimation, the model allows the selected word to create a new sense when all Q-values of its existing senses are lower than 0.5. Therefore, the prediction layer is initialized to 0 so that all Qvalues remain σ(0) = 0.5 until having been selected due to independent updates in Q-learning. Figure 2 illustrates the sense discovery procedure, where the model expands a word’s sense set when its context has all Q-values of existing senses less than 0.5.
3.4 Word Sense Exploration
Due to high ambiguity in natural language, a greedy sense selection strategy may not work well in the early training stage, because the sense selection module does not learn well. Such a drawback also exists in the literature using clustering algorithms (Neelakantan et al., 2014; Kågebäck et al., 2015) and hard-EM algorithms (Qiu et al., 2016; Jauhar et al., 2015). This issue is known as exploration-exploitation trade-off (Sutton and Barto, 1998). Li and Jurafsky (2015) proposed to sample on a Chinese Restaurant Process to introduce uncertainty. In our neural network architecture for computing Q-values, we perform dropout (Srivastava et al., 2014) in the hidden layer ∑ j∈C̄t Pj for our sense selection module to introduce uncertainty.
3.5 Sense Selection Prior
Incorporating priors in sense selection is a key element for probabilistic models (Li and Juraf-
sky, 2015; Jauhar et al., 2015), most previous work used online selection counts of each sense as its prior. As the key utility of the prior is to incorporate the sense selection preference for a word wi (disregarding its context {Ct−m, ..., Ct−1, Ct+1, ..., Ct+m}) in its sense selection process, we achieve similar prior utility by adding the target word wi into the input of the sense selection module as C̄t = {Ct−m, ..., Ct+m} such that the Q-values indicate the sense preference of the target word.
4 Experiments
We evaluate our proposed DRL-Sense model and compare with other multi-sense word representation models for both quantitative and qualitative experiments.
4.1 Experimental Setup and Configuration
Our model is trained on the April 2010 Wikipedia dump (Shaoul and Westbury, 2010), which contains approximately 1 billion tokens. For fair comparison, we adopt the same vocabulary set as Huang et al. (2012) and Neelakantan et al. (2014). For preprocessing, we convert all words to lower case, apply the Stanford tokenizer and the Stanford sentence tokenizer (Manning et al., 2014), and remove all sentences with less than 10 tokens.
In the experiments, the context window size is set to 6 (|C̄t| = 13). Subsampling technique introduced by word2vec (Mikolov et al., 2013b) is applied to accelerate the training process. The (upper bound of) number of senses per word in Q is set to 3 for fair comparison with prior work (Neelakantan et al., 2014). The learning rate is set to 0.025. Dropout is annealed from 0.5 to 0.0 within the first epoch. The embedding dimension is 300. We initialize Q and V as zeros, and P and U as the uniform distribution [− √ 1/100, √ 1/100] such that each embedding has unit length in expectation (Lei et al., 2015). For the negative sampling distribution in (1), we use 1/3 unigram of a word for each of its sense to compute the negative sampling probability using the 3/4rd power trick in Mikolov et al. (2013b). Our model uses 5 negative senses for negative sampling in (1).
In optimization, we conduct mini-batch training with 2048 batch size using the following procedure: 1) select senses in the batch; 2) optimize U, V using stochastic training within the batch for efficiency; 3) optimize P,Q using mini-
6
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
Method MaxSimC AvgSimC Huang et al. (2012) 26.1 65.7 Neelakantan et al. (2014) 60.1 68.6 Tian et al. (2014) 63.6 65.4 Li and Jurafsky (2015) 66.4 - Bartunov et al. (2016) 53.8 61.2 Qiu et al. (2016) 64.9 66.1 Proposed: DRL-Sense 66.6 65.2 - Non-Parametric 65.1 67.0 - Sense Exploration 66.2 64.5 - Sense Selection Prior 66.3 66.0
Table 1: Spearman’s rank correlation ρ x100 on the SCWS dataset.
batch training for robustness. To further stabilize the reward signal for the sense selection module, we only use the collocated sense σ(UTzik , Vzjl) in (1) to approximate the collocation likelihood Co(zik, zjl | U, V ) for Q-learning in (4).
4.2 Experiment 1: Contextual Word Similarity
To evaluate the quality of sense embeddings, we compute the similarity score between each word pair given their respective local contexts and compare with the human-judged score using Stanford’s Contextual Word Similarities (SCWS) data (Huang et al., 2012). Specifically, given a list of word pairs with corresponding contexts, S = {(wi, C̄t, wj , C̄t′)}, we calculate the Spearman’s rank correlation ρ between human-judged similarity and model similarity estimations. Two major contextual similarity estimations are introduced by Reisinger and Mooney (2010): AvgSimC and MaxSimC (also referred to LocalSim in Neelakantan et al. (2014)). AvgSimC is a soft measurement that addresses the contextual information with a probability estimation:
AvgSimC(wi, C̄t, wj , C̄t′) = Zi∑ k=1 Zj∑ l=1 p(zik|C̄t)p(zjl|C̄t′)d(zik, zjl),
which weights the similarity measurement of each sense zik and zjl by their probability estimations. On the other hand, MaxSimC is a hard measurement that only evaluates on the most probable senses:
MaxSimC(wi, C̄t, wj , C̄t′) = d(zik, zjl),
zik = arg max k′ p(zik′ |C̄t), zjl = arg max l′ p(zjl′ |C̄t′).
The baselines for comparison include classic clustering methods (Huang et al., 2012; Neelakantan et al., 2014), EM algorithms (Tian et al., 2014; Qiu et al., 2016; Bartunov et al., 2016), and Chinese Restaurant Process (Li and Jurafsky, 2015), where all approaches are trained on the same corpus except Li and Jurafsky (2015)3 and Qiu et al. (2016) used more recent Wikipedia dumps. The embedding sizes of all baselines are 300, except 50 in Huang et al. (2012). In our method, the probability assignment on each sense p(zik|C̄t) is calculated by replacing the sigmoid layer with a softmax layer in our sense selection module as:
∑
4.2.1 Embedding Quality Analysis
Our DRL-Sense model achieves the state-of-theart performance on MaxSimC, demonstrating superior quality on independent sense embeddings. In real world applications, using multiple sense vectors for a word simultaneously may bring additionally computational overhead over conventional single word embedding scenarios, and also change the existing neural network architecture for word embeddings. In contrast, by simply replacing the most probable sense identity with each word identity (as in MaxSimC), the computation framework and cost for downstream NLP tasks remains the same as conventional word embedding approaches. Rothe and Schütze (2015) reported that the traditional single word representation model word2vec (Mikolov et al., 2013b), released by Google and trained on 100 billion tokens Google’s internal dataset, can achieved 66.6% similarity correlation on the SCWS dataset. Table 1 shows that our proposed DRL-Sense achieves comparable performance as word2vec using only 1/100 size of Google’s internal data, while using multi-sense representations in the single embedding scenario (MaxSimC).
3We use Li and Jurafsky (2015)’s result on 1 billion tokens for fair comparison.
4More detail is available in Appendix B.
7
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
Method ESL-50 RD-300 TOEFL-80 (1) Conventional Word Embedding GC 47.73 45.07 60.87 SG 52.08 55.66 66.67 (2) Word Sense Disambiguation IMS+SG 41.67 53.77 66.67 (3) Direct Learning from Corpus EM 27.08 33.96 40.00 NP-MSSG 45.24 50.00 81.16 DRL-Sense 54.76 60.71 72.46 (4) Retrofitting with WordNet Retro-GC 63.64 66.20 71.01 Retro-SG 56.25 65.09 73.33
Table 2: Accuracy on synonym selection
4.2.2 Probability Assignment Analysis
To interpret the performance between MaxSimC and AvgSimC, we claim that an ideal model should have MaxSimC≥AvgSimC in Table 1. The reason is that, any sense selection model seeks a function f(zik | C̄t) to evaluate the fitness of each sense zik under a context C̄t, such as the cluster similarity in clustering methods, the likelihood or vanilla probability estimation in probabilistic models, and the Q-value in our model. Hence, to achieve the optimal expected fitness E[f(zik | C̄t)], the optimal probability assignment on senses is only selecting one and the optimal sense:
popt(zik | C̄t) = { 1 if k = arg maxk′ f(zik′ |C̄t), 0 otherwise.
Note that the above probability assignment makes AvgSimC=MaxSimC. Because the expected fitness E[f(zik | C̄t)] over any probability distribution would be no greater than the fitness value over the above distribution, the similar phenomenon (MaxSimC≥AvgSimC) can be observed in the robust fitness estimation (cluster similarity/likelihood/Q-value). Therefore, from Table 1, our DRL-Sense model demonstrates not only superior performance under a single sense embedding scenario but also reasonable and robust sense selection.
4.3 Experiment 2: Synonym Selection
We further evaluate our model on synonym selection using multi-sense word representations (Jauhar et al., 2015). Three standard synonym selection datasets, ESL-50 (Turney, 2001), RD-300 (Jarmasz and Szpakowicz, 2004), and
TOEFL-80 (Landauer and Dumais, 1997), are performed. In the datasets, each question consists of a question word wQ and four answer candidates {wA, wB, wC , wD}, and the goal is to select the most semantically synonymous choice among the four candidates. For example, in the TOEFL-80 dataset, a question shows {(Q) enormously, (a) appropriately, (b) uniquely, (c) tremendously, (d) decidedly}, and the answer is (c).
In the experiments, our model selects the synonym of the question word wQ by the collocation likelihood as a proxy of their semantic similarity. That is, for pairs of words (wQ, wT ), T ∈ {A,B,C,D}, we select the pair with the maximum bi-directional collocation likelihood,
Sim(wQ, wT )
= max j∈ZQ,l∈ZT max(Co(zQj , zT l), Co(zT l, zQj))
= max j∈ZQ,l∈ZT
max(σ(UTQjVT l), σ(U T T lVQj)).
Our model is compared with following baselines: (1) conventional word embedding: global context vectors (GC) (Huang et al., 2012) and skip-gram (SG) (Mikolov et al., 2013b); (2) applying supervised word sense disambiguation (using the IMS system (Zhong and Ng, 2010)) and then applying SG on disambiguated corpus: IMS+SG; (3) direct training multi-sense word representations on corpus: EM algorithm (Jauhar et al., 2015), non-parametric multi-sense skipgram (NP-MSSG) (Neelakantan et al., 2014) and our DRL-Sense; (4) retrofitting existing word embeddings to sense embeddings using WordNet: retrofitting GC (Retro-GC) and retrofitting SG (Retro-SG) (Jauhar et al., 2015). Note that we do not compare with the methods in (4) on Table 1, because sense representations are generated without a sense selector, and thus cannot be evaluated using MaxSimC and AvgSimC. Also, the methods in (4) can be treated as an upperbound of the methods in (3) due to the usage of additional supervised information from WordNet.
The experimental results are shown on Table 2. Except for retrofitting methods and NP-MSSG in TOEFL-80, our DRL-Sense model significantly outperforms all baselines. In addition, our model also performs better than Retro-GC for TOEFL80, although Retro-GC uses the supervised knowledge from WordNet, showing that our model obtains good sense embeddings by learning with the robust sense selector. Moreover, our method out-
8
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
Sense k-NN Senses mercury-1 mercury comet apollo comets wnba mercury-2 mercury sulfide beryllium zirconium apple-1 apple blackberry macintosh ibm novell apple-2 avocado apple apricot blackberry prunus lucky-1 lucky gotti co-starring gravano starred lucky-2 beyonce singin beenie baby choo lil lucky-3 hilfiger starbucks t-bone puff armani
Table 3: Sense representations with neighboring words using the collocation likelihood.
performs the EM method and NP-MSSG (Neelakantan et al., 2014), despite the highest score of AvgSimC on Table 1, echoing the superior quality of our sense vectors as suggested in the MaxSimC measurement.
4.4 Qualitative Analysis
We qualitatively evaluate non-parametric learning and sense representation learning performance.
4.4.1 Non-Parametric Learning Effectiveness
The largest challenge of multi-sense representation learning is that most words can be represented by a single embedding (non-polysemous or a single embedding can model multiple senses well). Hence, the feasible solution is to retain most words a single sense embedding, while leaving polysemous words multiple sense embeddings. Huang et al. (2012) and Neelakantan et al. (2014) only selected approximately 6,000 and 30,000 words from a vocabulary with size ≈ 100,000 for multisense representation learning. Qiu et al. (2016) utilized a coarse version (Navigli et al., 2007) of WordNet inventory (Miller, 1995) to determine the sense number for each word before training. The sense selection module in our learned DRL-Sense produces 71,194 words with a selected sense, 22,271 words with 2 senses, and 5,683 words with 3 senses based on the training data. Without any multi-sense determination procedure prior to training, our method automatically determines only about 30,000 polysemous words, where the number is close to Neelakantan et al. (2014)’s setting, demonstrating the effectiveness of our nonparametric learning algorithm.
4.4.2 Representation Learning Effectiveness
To evaluate the quality of sense embeddings, we show the k-nearest neighbors (k-NN) for each sense. For example, given the first sense of the word “apple”, denoted as “apple-1”, we select
Figure 3: The 2-D visualization using t-SNE (Maaten and Hinton, 2008) for the words “mercury” and “apple” on multi-sense word representations with L2 distance neighbors.
k senses with the highest collocation likelihood from the sense representation module and list in
Table 35. The learned sense embeddings of the words “mercury” and “apple” clearly correspond to different senses: “planet” (mercury-1), “metal element” (mercury-2), “computer” (apple-1), and “fruit” (apple-2) as visualized in Figure 3. Instead of actual senses, Table 3 also shows that our model is able to automatically detect a finegrained aspect division within a word “lucky” in an unsupervised manner. Specifically, it divides the semantics of “lucky” into three aspects: “starring” (lucky-1), “musician” (lucky-2), and “brands” (lucky-3), expanding the potential usage of the learned embeddings6.
5 Conclusion
We propose DRL-Sense—a novel deep reinforcement learning framework to jointly learn a word sense selection module and a sense representation module. Our model implements non-parametric learning for word sense induction and exploration for word sense selection. The experiments show that our DRL-Sense model achieves the state-ofthe-art performance for the benchmark contextual word similarity task and most of synonym selection datasets under the same setting. In the future, we plan to investigate reinforcement learning methods to incorporate multi-sense word representations for downstream NLP tasks.
5For clarity purpose, we omit the sense identity in the kNN senses.
6Note that the sense induction procedure highly depends on the training corpus, so it may fail to discover all senses in an existing sense inventory.
9
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
A Doubly Stochastic Gradient
For clarity purpose, we abbreviate the collocation log likelihood logCo(ik, jl|U, V ) as L(·) in this section. To derive doubly stochastic gradient for equation (2), we first resolve the expectation form as:
J(U, V, θ)
= Ek∼πiθ(·)[El∼πjθ(·)[L(·)]] = ∑ k ∑ l πiθ(zik|C̄t)π j θ(zjl|C̄t′)L(·).
The gradient of J with respect to Θ should be:
∂J(U, V, θ)
∂Θ
= ∂
∂Θ ∑ k ∑ l πiθ(zik|C̄t)π j θ(zjl|C̄t′)L(·)
= ∑ k ∑ l πiθ(zik|C̄t)L(·) ∂πjθ(zjl|C̄t′) ∂Θ
+ ∑ k ∑ l πjθ(zjl|C̄t′)L(·) ∂πiθ(zik|C̄t) ∂Θ
= Ek∼πiθ(·)[ ∑ l L(·) ∂ log πjθ(zjl|C̄t′) ∂Θ πjθ(zjl|C̄t′)]
+ E l∼πjθ(·) [ ∑ k L(·) ∂ log πiθ(zik|C̄t) ∂Θ πiθ(zik|C̄t)]
= Ek∼πiθ(·)[El∼πjθ(·)[L(·) ∂
∂Θ (log πiθ(zik|C̄t) + log π j θ(zjl|C̄t′))]]
Accordingly, if we conduct typical stochastic gradient ascent training on J(U, V,Θ) with respect to Θ from samples k and l with a learning rate η, the update formula will be:
∆Θ = ηL(·) ∂ ∂Θ (log πiθ(zik|C̄t) + log π j θ(zjl|C̄t′)).
However, because the collocation likelihood is a valid probability distribution, the collocation log likelihood should always be non-positive: L(·) = logCo(zik, zjl|U, V ) ≤ 0. Therefore, as long as the collocation log likelihood L is negative, the update formula is to minimize the likelihood of choosing l and k, despite the fact that l and k may be good choices. On the other hand, if the log likelihood reaches 0, it also indicates that the likelihood reaches infinity and computational overflow on U and V .
B Supplementary Experiments
In this section, we conduct ablation experiments by removing each components separately in our system to test the efficacy of proposed mechanisms. Specifically, we test the following components:
• Non-parametric: To remove the nonparametric learning mechanism in our model, we initialize the prediction layer Q in the sense selection module from uniform distribution [− √ 1/100, √ 1/100] as the
word embedding P .
• Sense exploration: To remove the word sense exploration mechanism in our model, we turn off the dropout mechanism during training.
• Sense selection prior: To remove the sense selection prior mechanism in our model, we remove the target word wi in the input of the sense selection module as C̄t = {Ct−m, ..., Ct−1, Ct+1, ..., Ct+m}.
In addition, we also test the efficacy of specific module according to their complexity. We test the setting when the embedding dimension in the sense selection module is set to 50 (50D/300D), and the embedding dimensions in the both modules are set to 50 (50D/50D).
The experiment results are shown in Table 4. Consistent improvement in terms of MaxSimC from each component to the proposed DRL-Sense model can be observed. In addition, the improvement from increasing the complexity (embedding dimension) demonstrate the efficacy of both modules (50D-50D→50D-300D and 50D300D→DRL-Sense).

TMP Strength: The paper propose DRL-Sense model that shows a marginal improvement on SCWS dataset and a significant improvement on ESL-50 and RD-300 datasets.  Weakness: The technical aspects of the paper raise several concerns: Could the authors clarify two drawbacks in 3.2? The first drawback states that optimizing equation (2) leads to the underestimation of the probability of sense. As I understand, eq(2) is the expected reward of sense selection, z_{ik} and z_{jl} are independent actions and there are only two actions to optimize. This should be relatively easy. In NLP setting, optimizing the expected rewards over a sequence of actions for episodic-task has been proven doable (Sequence Level Training with Recurrent Neural Networks, Ranzato 2015) even in a more challenging setting of machine translation where the number of actions ~30,000 and the average sequence length ~30 words. The DRL-Sense model has maximum 3 actions and it does not have sequential nature of RL. This makes it hard to accept the claim about the first drawback.  The second drawback, accompanied with the detail math in Appendix A, states that the update formula is to minimize the likelihood due to the log-likelihood is negative. Note that most out-of-box optimizers (Adam, SGD, Adadelta,  u2026) minimize a function f, however, a common practice when we want to maximize f we just minimize -f. Since the reward defined in the paper is negative, any standard optimizer can be use on the expected of the negative reward, which is always greater than 0. This is often done in many modeling tasks such as language model, we minimize negative log-likelihood instead of maximizing the likelihood. The authors also claim that when  the log-likelihood reaches 0, it also indicates that the likelihood reaches infinity and computational flow on U and V d (line 1046-1049). Why likelihood u2192infinity? Should it be likelihood u21921?  Could the authors also explain how DRL -Sense is based on Q-learning? The horizon in the model is length of 1. There is no transition between state-actions and there is not Markov-property as I see it (k, and l are draw independently). I am having trouble to see the relation between Q-learning and DRL-Sense.  In (Mnih et al., 2013), the reward is given from the environment whereas in the paper, the rewards is computed by the model. What  the reward in DRL-Sense? Is it 0, for all the (state, action) pairs or the cross-entropy in eq(4)?    Cross entropy is defined as H(p, q) = -  sum_{x} q(x)  log q(x), which variable do the authors sum over in (4)? I see that q(C_t, z_{i, k}) is a scalar (computed in eq(3)), while Co(z_{ik}, z_{jl}) is a distribution over total number of senses eq(1). These two categorial variables do not have the same dimension, how is cross-entropy H in eq(4) is computed then?  Could the authors justify the dropout exploration? Why not epsilon-greedy exploration? Dropout is often used for model regularization, preventing overfitting. How do the authors know the gain in using dropout is because of exploration but regularization?  The authors states that Q-value is a probabilistic estimation (line 419), can you elaborate what is the set of variables the distribution is defined? When you sum over that set of variable, do you get 1? I interpret that Q is a distribution over senses per word, however  definition of q in eq(3) does not contain a normalizing constant, so I do not see q is a valid distribution. This also related to the value 0.5 in section 3.4 as a threshold for exploration. Why 0.5 is chosen here where q is just an arbitrary number between (0, 1) and the constrain   sum_z q(z) = 1 does not held? Does the authors allow the creation of a new sense in the very beginning or after a few training epochs? I would image that at the beginning of training, the model is unstable and creating new senses might introduce noises to the model.  Could the authors comment on that?  General discussion What  the justification for omitting negative samples in line 517? Negative sampling has been use successfully in word2vec due to the nature of the task: learning representation. Negative sampling, however does not work well when the main interest is modeling a distribution p() over senses words. Noise contrastive estimation is often preferred when it comes to modeling a distribution. The DRL-Sense, uses collocation likelihood to compute the reward, I wonder how the approximation presented in the paper affects the learning of the embeddings.  Would the authors consider task-specific evaluation for sense embeddings as suggested in recent research [1,2]  [1] Evaluation methods for unsupervised word embeddings. Tobias Schnabel, Igor Labutov, David Mimno and Thorsten Joachims.  [2] Problems With Evaluation of Word Embeddings Using Word Similarity Tasks . Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, Chris Dyer  --- I have read the response.


Neural Disambiguation of Causal Lexical Markers based on Context
Causation is a psychological tool of humans to understand the world and it is projected in natural language. Causation relates two events, so in order to understand the causal relation of those events and the causal reasoning of humans, the study of causality classification is required. Herein, we propose a neural network architecture for the task of causality classification. We claim that the encoding of the meaning of a sentence is required for the disambiguation of its causal meaning. Our results show that our claim holds, and we outperform the state-of-the-art.
1 Introduction
Causation is a psychological tool of humans to understand the world independently of language, and it is one of the principles involved in the construction of the human mental model of reality (Neeleman and van de Koot, 2012). Causal reasoning is the process of relating two events, namely cause and its effect. Following the words of Reinhart (2002), causal relations are imposed by humans on the input from the world, and the (computational) linguist’s task is to understand what is about language that enables speakers to use it to describe their causal perceptions.
There are different theories concerning how natural language approximates causation. Those theories based on generative semantics argue that the causal relation is encoded in the semantics of some verbs (Lakoff, 1970; McCawley, 1976) or in the syntactic structure of a sentence (Dowty, 1979; Ramchand, 2008). For instance, Pylkkänen (2008) argues that the causing event may be associated with the subject of the causal verb in a causal predicate. This principle is true in the sentence “Pe-
ter eventually killed John by hitting him with a hammer”, in which the event of Peter hitting John with a hammer caused the death of John. But, the following sentence can be used as a counterexample: “A hammer eventually killed John”. In the later sentence the subject of “killed” is the “hammer”, which does not constitute a causing event for the event “kill John”. Therefore, the syntaxgrounded construction of causality defined by those theories is far away from the human mental model of causation. Indeed, Neeleman and van de Koot (2012) showed that the later approaches cannot be proven by standard syntactic tests, because neither the causing event nor causal relation correspond to a syntactic constituent. Neeleman and van de Koot (2012) concluded that the linguistic approximation of causation by culmination of events is not exclusive of causal verbs, and it is also found in non-causative verb classes.
Causation or causality has also been studied in computational linguistics. There are some semantic and discourse resources that take into account causality in the range of linguistic phenomena that they annotate. PropBank (Palmer et al., 2005) is a semantic resource that annotates the argument structure of verbs. The type of causation annotated in PropBank is related to the syntax and semantics of the verbs. For the sentences John broke the window and “The hammer broke the window”, “John” and “the hammer” are the cause arguments of the verb “broke”. Although, “John” and “the hammer” are required arguments for the event expressed by “break”, they do not represent the causing event of the window breaking event.
Causality is one of the discourse relations in Penn Discourse Tree Bank (PDTB) (Prasad et al., 2008). Causality is annotated in PDTB as either an explicit or an implicit relation between events. When causality is explicit, it is signalled by a lexical marker such as because or since between others.
2
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
However, there are some scenarios where the causal discourse relation is not signalled by a known lexical marker, but by other expressions that are called Alternative Lexicalizations (AltLex). The existence of AltLex means that the expression of causality is not defined by a limited number of lexical constructions, so the coverage of the total number of possible causal expressions is restricted to the grade of coverage of the base corpus of PDTB.
The previous paragraphs and the works described in Section 2 show that causality is driven by a limited set of lexical elements and syntactic constructions. However, causality does not have a fixed list of lexical or syntactic constructions. Hidey and McKeown (2016) stressed the issue of the limited coverage of causal expressions and released a corpus with a larger amount of causal expressions than PDTB. They also proposed a classification method, which is based on the use of features from the corpus and features from lexical resources. The last set of features reduces the ability of the system to classify causality, because they restrict the system to the causal definition of the lexical resources.
Causal meaning disambiguation is the task of identifying whether there is a causal relation between two events. We hypothesize that neural networks are able to encode the meaning of those events, and discover whether the underlying relation between them is causal. Herein, we contribute a neural network architecture for the task of causality disambiguation and we assess it in the corpus of Hidey and McKeown (2016). Empiric results on this corpus show that our claim indeed holds.
2 Related Work
Khoo et al. (1998) presented one of the first works related to the classification of causal relations. The authors defined a set of linguistic rules for the identification of cause-effect relations, and developed a pattern matching system based on those rules. The errors of the system show the common errors of a full linguistically grounded classification system, which are related to the ambiguity of some causal links such as by or as or the lexical and syntactic ambiguity of some causative verbs. Those errors are related to the lack of evidence of the syntax basis of causation (Neeleman and van de Koot, 2012).
Girju (2003) proposed a decision tree learning method to classify causal relations between events. The type of relations studied by the author are those conveyed by a verb and two noun phrases. The clas-
sification method uses as lexical features whether the verb of the sentence belongs to a list of ambiguous causative verbs, and whether the nouns belongs to one of the nine semantic hierarchies in WordNet. Again, the main source of errors is the ambiguity of the projection of causality in natural language.
Bethard and Martin (2008) defined a method for the classification of causal relations between verbal events in conjunction constructions, namely conjoined events. Since the method was defined for a specific kind of causal relation between events, the authors had to first create a corpus following that requirement. The authors developed a supervised classification system using syntactic and semantic features from WordNet, and they used SVM as a classification algorithm. The drawback of the method is its reduced flexibility, because it is only defined for a specific causal construction and its confidence in syntactic features. Moreover, the authors conclude that semantic features are more adequate for causality classification than syntactic features, which goes in the direction of our claim.
The first attempt of encoding the contextual information for causality classification between events is found in Do et al. (2011). The authors defined a measure of the causality relation between two events according to the co-occurrence frequency between the predicates and their arguments. The measure is inspired by the causality score defined by Riaz and Girju (2010).
Following the work of Do et al. (2011), Riaz and Girju (2013, 2014) built a knowledge base for causal events, first for events related by a pair of verbs, and then for events conveyed by pairs of verbs and noun phrases. In these works, the authors attempted to go in the direction of using distributional semantics in order to encode the causal meaning, but they still rely on the use of lexical and syntactic features. Thus, the main drawbacks of those papers are the complex featuring engineering process to detect the causal relation, and the restricted coverage of the resulting knowledge bases.
Following the previous approaches, Mirza and Tonelli (2016) described a causal classification system built upon a rule based system and a machine learning algorithm based on lexical, syntactic and semantic features. The novelty lies in the use of features of temporal relations. The work of Mirza and Tonelli (2016) has similar issues to the work of (Bethard and Martin, 2008), because they use similar features to represent causality.
3
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
Sentence Type
(Cathay Pacific delayed both legs of its quadruple daily Hong Kong to London route)e1 ([due to]l this disruption in air traffic services.)e2 Explicit (The factory was not well equipped to handle the gas)e1 (created by the sudden addition of water to the MIC tank.)e2 Implicit
Table 1: Explicit and implicit causal relations
The previous works support the idea that causality is defined by a specific syntactic construction or the semantics of verbs known as causative verbs. On the other hand, some of the works restricted their study to a narrow kind of syntactic constructions, such as the conjoined events of Bethard and Martin (2008). In contrast, we argue that the representation of causality should not be limited by syntactic patterns or the coverage of semantic resources, due to the lack of a particular linguistic construction for causality. Therefore, we propose the neural encoding of the context of the relation in order to disambiguate its causal meaning.
3 Causality classification
The next sections expose the definition of the task (Section 3.1), the data used in the experiments (Section 3.2) and the proposed system (Section 3.3).
3.1 Definition
Causality is defined as the semantic relation between two events (e1, e2), namely causing event (cause, e1) and caused event (effect, e2). An event may be a verb, whose arguments may be explicitly present or not (subject, verb, object), or a noun phrase. The relation can be explicit or implicit, in case it is explicitly expressed, it is signalled by a lexical marker (l), which may be a verb, a preposition, an adverb or an expression such as as a result of, see Table 1.
Some researchers impose two additional restrictions: the temporal restriction, which means that the causing event should take place before the caused event, and the counterfactual one, which says if the causing event did not occur, the caused event would not have occurred either. Thus, causation can be formally defined as e1 → e2. An example from the test set used in the experiments that follows the three restrictions is the following:
A government affidavit in 2006 stated that (the leak)e1 (caused 558,125 injuries, including 38,478 temporary partial injuries and approximately 3,900 severely and permanently disabling injuries)e2
It is evident that e1 is before e2 and if the leak did not occur, the set of events in e2 would not occur. Nevertheless, we support the same opinion as Neeleman and van de Koot (2012) and we do not add the counterfactual constraint in our definition, because it is not true in all the cases. For example:
(The argument between John’s parents)e1 (broke the window)e2
If we follow the counterfactual constraint, the event of the argument between the John’s parents should be the causing event. However, the human mental model of causality does not usually relate an argument with the breaking of a window. Causation is concerned with the human understanding of the world and not with the world itself.
Concerning the temporal restriction, it should be interpreted as a physical ordering constraint and not as a positional one. That means, the causing event does not always appear before the caused event in the sentence. The caused event (e2) is before the causing event (e1) in the following example:
(This water was diverted)e2 ([due to]l a combination of improper maintenance, leaking and clogging, and eventually ended up in the MIC storage tank.)e1
Some related work attempted to identify the causing, the caused event and its direction (Mirza and Tonelli, 2016), but for that purpose, they restricted the study to a specific syntactic construction. We claim that the task of causality classification has not to be restricted to a syntactic construction and it has to be set up as a two steps task: causal meaning classification and causal arguments identification. The task of causal meaning classification is a binary classification task that is defined as the disambiguation of the causal meaning of the relation of two events. The input of the task is two events and the output is the meaning of the relation (Causal or Non Causal). The task of causal argument identification is focused on the identification of the causing (e1) and caused (e2) events. The aim of this paper is to contribute to the first subtask, so given two events (e1, e2) and the expression susceptible of signalling causality (l), the system returns the meaning of the relation (Causal or Non Causal). The following sentence is an example of the input of our system.
(He fell in love with her and changed his life)e1 ([so]l he could help her)e2
4
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
Sentence Meaning
An undercroft is traditionally a cellar or storage room, often brick-lined and vaulted, and used for storage in buildings since medieval times
Temporal
Additionally if one is to use a large scan range then sensitivity of the instrument is decreased due to performing fewer scans per second since each scan will have to detect a wide range of mass fragments
Causal
In stark contrast to his predecessor, five days after his election he spoke of his determination to do what he could to bring peace Temporal Bischoff in a round table discussion claimed he fired Austin after he refused to do a taping in Atlanta Causal
Table 2: Examples of different meanings of the prepositions since and after taken from the AltLex corpus
3.2 Data
Causality may be signalled by a verb (cause), a preposition (because), an adverb (subsequently) or an expression such as as a result of. Some of those causal expressions are almost unambiguous, but the causal meaning of others totally depends on the context. For instance, according to PDTB, the preposition because is an unambiguous lexical marker of causation, but there are other expressions whose causal meaning is not unarguable, such as since or after that may have a temporal or a causal interpretation, see Table 2. Therefore, we need a corpus in which the events, the lexical marker and the meaning of the relation are annotated.
To the best of our knowledge, there are three available corpora. The first one was exposed in Bethard and Martin (2008) and it is built on top of the Penn TreeBank corpus. The corpus only provides the position of the related events in each sentence and whether the relation is causal. Unfortunately, the corpus is not freely available, and is also not large enough to train neural methods.
The second corpus is Causal-TimeBank (Mirza et al., 2014), which is built on top of the TimeML corpus. Annotation was restricted to explicit causal relations. They followed the same annotation schema of TimeML, so they defined the label CSIGNAL for the causal lexical markers, and CLINK to mark the causal relation between two events. The authors manually annotated the causal relations between the identified events in TimeML. The fact that the causal lexical markers are annotated agrees with our data requirements, but as the previous corpus, Causal-TimeBank is too small to train neural methods.
Recently, the AltLex corpus has been freely released (Hidey and McKeown, 2016). The corpus was built on the idea that causation can be expressed by different kinds of linguistic constructions. This is validated by the fact that in PDTB there are explicit causal lexical markers, and other sort of expressions that have a discourse meaning,
which are called AltLex. The relations signalled by an AltLex expression are implicit relations, in which the annotator did not find an appropriate connective to insert between the events, because the meaning of the relation is entailed by other expressions, namely AltLex. Furthermore, some causal lexical markers can be perceived as variations of the basic ones, but those variations are not usually in the list of discourse connectives. Thus, the authors of the AltLex corpus decided to develop a method to identify a larger amount of AltLex expressions with causal meaning. The corpus construction leveraged Simple Wikipedia, by aligning sentences from Wikipedia that consist of unknown lexical causal markers with sentences from Simple Wikipedia that contain corresponding known lexical causal markers. Once a first set of causal and non-causal sentences were identified, a bootsrapping method was applied to enhance the corpus. We call “non bootstrapping” to the first version of the corpus and “bootstrapping” to the second one. The corpus size is shown in Table 3. For further details see (Hidey and McKeown, 2016).
The class distribution of the lexical markers should be studied, because one of the features of the corpus is the annotation of the lexical markers susceptible of expressing causation. Table 4 shows the class distribution of the lexical markers, as well as the the number of unarguable ones and the number of lexical markers with mostly a different meaning in the training and the test set. According to the Table 4, there are few ambiguous connectives, 121 in the “non bootstrapping” corpus and 147 in the “bootstrapping” version. However, there is an important difference between the two versions of
5
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
Bootstrapping
No Yes
Total 8214 8854 Unambiguous in Causal class 922 1034 Unambiguous in Non Causal class 7171 7673 Causal in train, Non Causal in test 0 27 Non Causal in train, Causal in test 0 8
Table 4: Distribution of the lexical markers
the corpus: there are no differences between the training and the test set in the “non bootstrapping” version, but there are in the “bootstrapping” one. This fact means that the “bootstrapping” version of the corpus has some instances that do not follow the class distribution of the training set, so they represent a higher difficulty for the classification system. Section 4.3 exposes the good performance of the proposed system in some of those instances.
3.3 Disambiguation of the Causal Meaning
Kruengkrai et al. (2017) describe a Multi-column Convolutional Neural Network (CNN) that uses background knowledge for the identification of the causal meaning of an input sentence in Japanese. Convolutional networks excel at pattern learning in input data, however causation does not have a particular syntactic structure as it was mentioned. Moreover, CNN requires the definition of the kernel size, which means that we should know beforehand the length of the context that projects the causal meaning of the sentence. Nevertheless, the list of expressions that may represent a causal meaning is not limited and some sentences implicitly express causality. On the other hand, Long short-term memory (LSTM) recurrent neural networks (RNN) (Hochreiter and Schmidhuber, 1997) can encode the sequential information of the input text. Melamud et al. (2016) also showed that using an LSTM results in a more precise context encoding and substantially improve performance in several tasks upon the common average-of-wordsembeddings representation. Therefore, we propose the use of LSTM RNN instead of a CNN, because we argue that LSTM has a higher capacity of encoding meaning for the task of causality classification.
We propose a neural network architecture with two inputs, which is mainly based on the encoding of the two inputs with an LSTM and the use of several dense layers with a tanh activation function.
Following the assumption that some sort of relation should exist between the two events of a causal relation, the first evaluated model consists of two
connected LSTMs, or in other words, the LSTM network of the second event of the causal relation is initialized with the last state of the first LSTM. We call to this architecture “Stated Pair LSTM”. We also evaluated the same architecture but without the connection between the two LSTMs. We call to this second architecture “Pair LSTM” and it is depicted in Figure 1.
According to Figure 1, the first input of the system is the first event of the relation, and the second input is the concatenation of the lexical marker and the second event. The text processing starts with the tokenization of the two inputs, and the representation of them as a matrix of word embedding vectors. The set of pre-trained embeddings used was Glove (Pennington et al., 2014), specifically the 300 dimension reference vectors of the 840B cased tokens set.
The lengths of the first (n) and the second (m) input are not the same, so three zero-padding strategies were evaluated to measure which of them is the most convenient to encode the causal meaning. The maximum, the mean and mode of the length of the two inputs were calculated:
∀x ∈ IRn×d and ∀y ∈ IRm×d
pad(x) = {max(x),mean(x),mode(x)} (1)
where x and y are the first and second input and d is the length of the word embedding vectors.
Subsequently, each of the two outputs of the encoding layer are vectorized to a vector of length 100 by a dense layer with a tanh activation function. The context of the causal relation is represented by the concatenation of the two vectors.
∀W ∈ IR100×nd and ∀ b ∈ IR100
vec(x) : IRn×d → IRnd
tanh(W · x+ b) context : (vec(x), vec(y))
(2)
6
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
The output of the Equation 2 is processed by three dense layers activated by a tanh function. The last layer is composed of the softmax operation.
The performance of two learning optimizers was evaluated, specifically Adadelta (Zeiler, 2012) and Adam (Kingma and Ba, 2015). In order to avoid the overfitting of the classification process, different values for dropout ([0.5, 0.75]) and regularization were used ( [ 8 · 10−3, 8 · 10−6 ] ).
4 Experiments and Results
As far as we know, the AltLex corpus has been only used in the work in which it was presented (Hidey and McKeown, 2016), therefore we consider their method as the state of the art on that corpus. Moreover, as it has been mentioned several times, the task of causality classification lacks of representative or large enough corpus that covers a wide range of causal expressions, as well as without the restriction of being composed of specific causal constructions. Therefore, the methods were only evaluated with the AltLex corpus. We have used the two versions of the corpus (“non bootstrapping” and “bootstrapping”) in our experiments. The size of each of them is shown in Table 3.
4.1 Baselines
We consider two baselines in order to compare our proposal. The first one (B1) is founded on the use of the most common class of each causal connective according to its class in the training data. For example, if the lexical marker since has mostly a causal meaning, B1 always classifies the relation signalled by since as Causal. A similar approach was also used in (Hidey and McKeown, 2016).
The second baseline (B2) is the system of Hidey and McKeown (2016), which is based on SVM with a large set of features generated from the combination of features from the original parallel corpus and some lexical resources (WordNet, VerbNet and PropBank). The fact of relying on lexical resources restricts the recall of the system to the linguistic coverage of the lexical resources. In contrast, we propose a neural network architecture fed only by a set of word embedding vectors, which has a higher ability of generalization as it is shown later in the paper.
4.2 Results
The results reached by our system are shown in
Table 5.1 The Precision, Recall and F1 values were used to measure the performance of the system in the Causal Class, and the Accuracy to measure the overall performance of the system.
The performance of B1 defines a hard baseline for the two versions of the corpus. On the other hand, the good performance of B1 might indicate that the training corpus is composed of few unambiguous causal connectives. Concerning the results reached by the proposed systems, all the configurations in Table 5 outperform the baseline B1, which means that the system learns beyond the class distribution of the lexical markers in the training data.
Those configurations that use Adadelta as optimizer outperform the system B2 in the “non bootstrapping” version of the corpus. The behaviour of B2 shows a big difference between Precision and Recall, indicating that the system has many false positives, i.e. a large number of sentences without a causal meaning. This is not a desirable behaviour. On the other hand, our system has a high Precision with lower Recall, indicating it mainly classifies correctly sentences with unambiguous connectives. Figure 2 shows that the most frequent connectives in the Causal class do not have any instance in the Non Causal class, which supports our last assertion. The best configuration (“Pair LSTM Max Adadelta”) uses the maximum operation for the zero-padding strategy and Adadelta as learning optimizer. Our proposal in this scenario improves the state of the art by 2.13% according to F1 in the Causal class (C) and 7.72% according to Accuracy.
We observe a similar trend in the performance of the system with the “bootstrapping” version of the corpus, i.e. the system architecture “Pair LSTM” tends to reach better results than the architecture “Stated LSTM”. This behaviour means that our assumption about the relation between the meanings of the two input events does not hold, so it is better to encode each argument independently and then to measure the relation between the arguments by using dense layers. Concerning the optimizers, the results show a similar behaviour in both architectures. The Adadelta optimizer promotes the Recall and penalizes Precision, whereas the system reaches a higher value of Precision and a smaller difference between Precision and Recall
1For the sake of brevity, those systems that performed worse than B1 and B2 are not in Table 5.
7
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
Corpus Method Precision C. Recall C. F1 C. Accuracy
Non bootstrapping B1 68.92% 54.92% 61.13% 63.99% B2 70.28% 77.60% 73.76% 71.86% Stated Pair LSTM Mean Adadelta 90.04% 60.31% 72.24% 76.10% Stated Pair LSTM Mode Adadelta 89.23% 63.17% 73.97% 77.08% Pair LSTM Max Adadelta 88.46% 65.71% 75.40% 77.90% Pair LSTM Mean Adadelta 89.33% 63.80% 74.44% 77.41%
Bootstrapping
B1 74.38% 86.66% 80.05% 77.74% B2 77.29% 84.85% 80.90% 79.58% Stated Pair LSTM Max Adadelta 78.69% 84.44% 81.47% 80.19% Stated Pair LSTM Mean Adam 80.00% 82.53% 81.25% 80.36% Stated Pair LSTM Mode Adam 80.24% 82.53% 81.37% 80.52% Pair LSTM Max Adadelta 78.07% 84.76% 81.12% 79.86% Pair LSTM Mean Adadelta 78.48% 85.71% 81.94% 80.52% Pair LSTM Mean Adam 80.30% 82.85% 81.56% 80.68% Pair LSTM Mode Adam 77.24% 87.30% 81.96% 80.19%
Table 5: Results of the baselines (B1 and B2) and the configurations evaluated
b e ca u se b e ca u se o f so so t h a t th u s su b se q u e n tl y a s a r e su lt o f th e re fo re h e n ce in r e sp o n se t o a s d u e t o th e re b y so i n so a s a n d fo r d u e a n d s o co n se q u e n tl y si n ce th e n a n d b e ca u se a ft e r so f o r w h e n so o n b e ca u se i n B e ca u se o f S o so i m p re ss e d w it h T h u s so a t th a t b e ca u se so w it h ca n t h e re fo re so b y so i m p re ss e d b y so m u ch t h a t b e ca u se a t th u s : T h e re fo re so b e ca u se so a ft e r so s u cc e ss fu l th a t a cc o rd in g ly in o rd e r ca n t h u s so f a r a s so f ro m
0
500
1000
1500
2000
F re q u e n cy
Causal Non Causal
Figure 2: Distribution of the most frequent connectives in the class Cause
when Adam optimizer is used, which is a more desirable behaviour. A high value of Precision and a high value of Recall mean that there is a good balance between the accuracy in the disambiguation of the causal meaning of the relation and the coverage of different ways of expressing causality. To conclude, the system that reached the highest performance is the one that used the mean strategy for zero-padding and the optimizer Adam.
Most of the configurations of the proposed neural network outperform the baseline B2. If we look at the values of Precision, our proposal improves the Precision value of B2 by 3.89% . However, B2 is better 2.41% according to the Recall value. The good Recall value of B2 system entails a drop in the performance according to the Precision. This behaviour is also developed by the neural network configurations that use maximum strategy for zero-padding and Adadelta as optimizer, but in those cases the Recall and the Pre-
cision are greater. As previously mentioned, this means that the systems have a large number of false positive instances, so they might not be correctly learning the characteristics of the Causal class. In contrast, “Pair LSTM Mean Adam” reached the best Precision in the Causal class and the less difference between Precision and Recall, thus it also reached a better balance between the accuracy in the disambiguation of the causal relation and the coverage of different ways of expressing causality.
Corpus Version Causal Non Causal Total
Training
Non bootstrapping
7,606 7,929 15,534
Bootstrapping 12,534 8,821 21,354
The class distribution of the AltLex corpus is not balanced (see Table 3), so we reduced the number of instances of the Non Causal by a factor of ten with the aim of evaluating our system with a more balanced dataset. The new corpus size is in Table 6. We assessed the performance of the baseline B1 and the system “Pair LSTM Mean Adam” and the results are in Table 7. Regarding the performance of B1 in the “non bootstrapping version”, the Precision is lower and the Recall is higher than in the previous experiments, which means that the number of false positives is large, so it is not learning the characteristics of the class Causal. As in the previous set of experiments, B1 promotes Recall and penalizes Precision in the “bootstrapping” version of the corpus, which is not a desirable behaviour. In contrast, our network architecture followed the same trend in both set of experiments and with the both version of the corpus. “Pair LSTM Mean Adam”
8
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
Corpus Method Precision C. Recall C. F1 C. Accuracy
Non bootstrapping B1 63.70% 84.12% 72.50% 67.10%Pair LSTM Mean Adam 76.38% 69,84% 72,96% 73.32%
Bootstrapping B1 67.34% 94.28% 78.57% 73.48%Pair LSTM Mean Adam 72.65% 88.57% 79.82% 76.92%
Table 7: Results of B1 and Pair LSTM Mean Adam with the reduced version of the AltLex corpus
Sentence Training Test B2 Our proposal
The United States decided to break off economic relations with Cuba (which means that they would stop buying things from them). Causal Non Causal Causal Non Causal
Although Roosevelt had promised to keep the United States out of the war, he nevertheless took concrete steps to prepare for war. Causal Non Causal Causal Non Causal
Greatly alarmed and with Hitler making further demands on the Free City of Danzig, Britain and France guaranteed their support for Polish independence; when Italy conquered Albania in April 1939, the same guarantee was extended to Romania and Greece. Causal Non Causal Causal Non Causal
One of these fragments gives rise to fibrils of amyloid beta, which then form clumps that deposit outside neurons in dense formations known as senile plaques. Ambiguous Causal Non Causal Causal
Table 8: Some correctly classified examples by our best configuration and misclassified by B2
significantly improves the performance of B1 according to the McNemar’s test with a P-value of 0.005 and 0.05 respectively.
4.3 Analysis
As mentioned in the previous sections, there are some linguistic constructions that can represent a causal meaning in some contexts, but not in other and vice-versa. A better balance between Precision and Recall in the Causal class means that the system learns the causal meaning better. Roughly speaking, the quality of the causal disambiguation is better, also for those lexical markers that are mostly Causal in the training data. We present some correctly classified examples by our best configuration and misclassified by B2 in Table 8. It shows the class of the instance in the test corpus and the value of the most frequent class in the training corpus. The behaviour of the verb break should be stressed since it is considered as a causative verb. However, there are some other uses of break that do not have a causal meaning (see Table 8).
The behaviour of the proposed system with the connective which then is also remarkable. That connective is totally ambiguous, because it only has one instance labelled as Causal and other as Non Causal in the training data. We have to take into account that we work with word embeddings, so if the individual words which and then are over-represented in one class, that fact can determine the behaviour of the classification system.
The individual term which does not appear in any instance of the training corpus, while the word then is mostly in sentences without a causal meaning. Despite the difficulty of the expression which then, we can see in Table 8 that our proposed system correctly disambiguate its causal meaning, while the system of Hidey and McKeown (2016) does not. Therefore, the results reached and those examples allow us to confirm our claim that the encoding of the context is required for the disambiguation of the causal meaning as we have shown in this paper.
5 Conclusions and Future Work
We defined the task of causation classification as a task composed of another two subtasks: causal meaning classification and causal argument classification. The paper was focused on the first subtask, and we claim that the encoding of the two events of the relation is required for a suitable disambiguation of causality. We proposed an encoding system based on a neural network with two inputs, one for the first event and the other for the lexical marker and the second event. Our proposed system outperforms the state-of-the-art. We also showed the success of the system in some non-causative sentences but with commonly causative verbs (see Table 8).
One of the problems of the task is the lack of resources, so, as future work, we plan the creation of a new corpus for the two subtasks of causality classification, namely causality disambiguation and causality argument classification.
9
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899

This paper develops an LSTM-based model for classifying connective uses for whether they indicate that a causal relation was intended. The guiding idea is that the expression of causal relations is extremely diverse and thus not amenable to syntactic treatment, and that the more abstract representations delivered by neural models are therefore more suitable as the basis for making these decisions.  The experiments are on the AltLex corpus developed by Hidley and McKeown. The results offer modest but consistent support for the general idea, and they provide some initial insights into how best to translate this idea into a model. The paper distribution includes the TensorFlow-based models used for the experiments.  Some critical comments and questions:  * The introduction is unusual in that it is more like a literature review than a full overview of what the paper contains. This leads to some redundancy with the related work section that follows it. I guess I am open to a non-standard sort of intro, but this one really doesn't work: despite reviewing a lot of ideas, it doesn't take a stand on what causation is or how it is expressed, but rather only makes a negative point (it's not reducible to syntax). We aren't really told what the positive contribution will be except for the very general final paragraph of the section.  * Extending the above, I found it disappointing that the paper isn't really clear about the theory of causation being assumed. The authors seem to default to a counterfactual view that is broadly like that of David Lewis, where causation is a modal sufficiency claim with some other counterfactual conditions added to it. See line 238 and following; that arrow needs to be a very special kind of implication for this to work at all, and there are well-known problems with Lewis's theory (see http:  bcopley.com wp-content uploads CopleyWolff2014.pdf). There are comments elsewhere in the paper that the authors don't endorse the counterfactual view, but then what is the theory being assumed? It can't just be the temporal constraint mentioned on page 3!  * I don't understand the comments regarding the example on line 256. The authors seem to be saying that they regard the sentence as false. If it's true, then there should be some causal link between the argument and the breakage. There are remaining issues about how to divide events into sub-events, and these impact causal theories, but those are not being discussed here, leaving me confused.  * The caption for Figure 1 is misleading, since the diagram is supposed to depict only the  Pair_LSTM  variant of the model. My bigger complaint is that this diagram is needlessly imprecise. I suppose it's okay to leave parts of the standard model definition out of the prose, but then these diagrams should have a clear and consistent semantics. What are all the empty circles between input and the  LSTM  boxes? The prose seems to say that the model has a look-up layer, a Glove layer, and then ... what? How many layers of representation are there? The diagram is precise about the pooling tanh layers pre-softmax, but not about this. I'm also not clear on what the  LSTM  boxes represent. It seems like it's just the leftmost final representation that is directly connected to the layers above. I suggest depicting that connection clearly.  * I don't understand the sentence beginning on line 480. The models under discussion do not intrinsically require any padding. I'm guessing this is a requirement of TensorFlow and or efficient training. That's fine. If that's correct, please say that. I don't understand the final clause, though. How is this issue even related to the question of what is  the most convenient way to encode the causal meaning ? I don't see how convenience is an issue or how this relates directly to causal meaning.  * The authors find that having two independent LSTMs ( Stated_LSTM ) is somewhat better than one where the first feeds into the second. This issue is reminiscent of discussions in the literature on natural language entailment, where the question is whether to represent premise and hypothesis independently or have the first feed into the second. I regard this as an open question for entailment, and I bet it needs further investigation for causal relations too. So I can't really endorse the sentence beginning on line 587:  This behaviour means that our assumption about the relation between the meanings of the two input events does not hold, so it is better to encode each argument independently and then to measure the relation between the arguments by using dense layers.  This is very surprising since we are talking about subparts of a sentence that might share a lot of information.  * It's hard to make sense of the hyperparameters that led to the best performance across tasks. Compare line 578 with line 636, for example. Should we interpret this or just attribute it to the unpredictability of how these models interact with data?  * Section 4.3 concludes by saying, of the connective 'which then', that the system can  correctly disambiguate its causal meaning , whereas that of Hidey and McKeown does not. That might be correct, but one example doesn't suffice to show it. To substantiate this point, I suggest making up a wide range of examples that manifest the ambiguity and seeing how often the system delivers the right verdict. This will help address the question of whether it got lucky with the example from table 8.