Neural network models have shown their promising opportunities for multi-task learning, which focus on learning the shared layers to extract the common and task-invariant features. However, in most existing approaches, the extracted shared features are prone to be contaminated by task-specific features or the noise brought by other tasks. In this paper, we propose an adversarial multi-task learning framework, alleviating the shared and private latent feature spaces from interfering with each other. We conduct extensive experiments on 16 different text classification tasks, which demonstrates the benefits of our approach. Besides, we show that the shared knowledge learned by our proposed model can be regarded as off-the-shelf knowledge and easily transferred to new tasks.
1 Introduction
Multi-task learning is an effective approach to improve the performance of a single task with the help of other related tasks. Recently, neuralbased models for multi-task learning have become very popular, ranging from computer vision (Misra et al., 2016; Zhang et al., 2014) to natural language processing (Collobert and Weston, 2008; Luong et al., 2015), since they provide a convenient way of combining information frommultiple tasks.
However, most existing work on multi-task learning attempts to divide the features of different tasks into private and shared spaces, merely based on whether parameters of some components should be shared. As shown in Figure 1- (a), the general shared-private model introduces two feature spaces for any task: one is used to
store task-dependent features, the other is used to capture shared features. The major limitation of this framework is that the shared feature space could contain some unnecessary task-specific features, while some sharable features could also be mixed in private space, suffering from feature redundancy. Taking the following two sentences as examples, which are extracted from two different sentiment classification tasks: Movie reviews and Baby products reviews.
The infantile cart is simple and easy to use. This kind of humour is infantile and boring.
The word “infantile” indicates negative sentiment in Movie task while it is neutral in Baby task. However, the general shared-private model could place the task-specific word “infantile” in a shared space, leaving potential hazards for other tasks. Additionally, the capacity of shared space could also be wasted by some unnecessary features. To address this problem, in this paper we propose an adversarial multi-task framework, in which the shared and private feature spaces are inherently disjoint by introducing orthogonality constraints. Specifically, we design a generic sharedprivate learning framework to model the text se-
2
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
quence. To prevent the shared and private latent feature spaces from interfering with each other, we introduce two strategies: adversarial training and orthogonality constraints. The adversarial training is used to ensure that the shared feature space simply contains common and task-invariant information, while the orthogonality constraint is used to eliminate redundant features from the private and shared spaces.
The contributions of this paper can be summarized as follows.
1. Proposed model divides the task-specific and shared space in a more precise way, rather than roughly sharing parameters. 2. We extend the original binary adversarial training to multi-class, which not only enables multiple tasks to be jointly trained, but allows us to utilize unlabeled data. 3. We can condense the shared knowledge among multiple tasks into an off-the-shelf neural layer, which can be easily transferred to new tasks.
2 Recurrent Models for Text Classification
There are many neural sentence models, which can be used for text modelling, involving recurrent neural networks (Sutskever et al., 2014; Chung et al., 2014), convolutional neural networks (Collobert et al., 2011; Kalchbrenner et al., 2014), and recursive neural networks (Socher et al., 2013). Here we adopt recurrent neural network with long short-term memory (LSTM) due to their superior performance in various NLP tasks.
Long Short-term Memory Long short-term memory network (LSTM) (Hochreiter and Schmidhuber, 1997) is a type of recurrent neural network (RNN) (Elman, 1990), and specifically addresses the issue of learning long-term dependencies. While there are numerous LSTM variants, here we use the LSTM architecture used by (Jozefowicz et al., 2015), which is similar to the architecture of (Graves, 2013) but without peep-hole connections.
We define the LSTM units at each time step t to be a collection of vectors in Rd: an input gate it, a forget gate ft, an output gate ot, a memory cell ct and a hidden state ht. d is the number of the LSTM units. The elements of the gating vectors it, ft and ot are in [0, 1].
The LSTM is precisely specified as follows.
 c̃t ot it ft  =  tanh σ σ σ (Wp [ xtht−1 ] + bp ) , (1)
ct = c̃t ⊙ it + ct−1 ⊙ ft, (2) ht = ot ⊙ tanh (ct) , (3)
where xt ∈ Re is the input at the current time step; Wp ∈ R4d×(d+e) and bp ∈ R4d are parameters of affine transformation; σ denotes the logistic sigmoid function and ⊙ denotes elementwise multiplication. The update of each LSTM unit can be written precisely as follows:
ht = LSTM(ht−1,xt, θp). (4)
Here, the function LSTM(·, ·, ·, ·) is a shorthand for Eq. (1-3), and θp represents all the parameters of LSTM.
Text Classification with LSTM Given a text sequence x = {x1, x2, · · · , xT }, we first use a lookup layer to get the vector representation (embeddings) xi of the each word xi. The output at the last moment hT can be regarded as the representation of the whole sequence, which has a fully connected layer followed by a softmax non-linear layer that predicts the probability distribution over classes.
ŷ = softmax(WhT + b) (5)
where ŷ is prediction probabilities, W is the weight which needs to be learned, b is a bias term. Given a corpus with N training samples (xi, yi), the parameters of the network are trained to minimise the cross-entropy of the predicted and true distributions.
L(ŷ, y) = − N∑ i=1 C∑ j=1 yji log(ŷ j i ), (6)
where yji is the ground-truth label; ŷ j i is prediction probabilities, and C is the class number.
3 Multi-task Learning for Text Classification
The goal of multi-task learning is to utilizes the correlation among these related tasks to improve
3
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
softmax Lmtask
LSTM
softmax Lntask
xm xn
(a) Fully Shared Model (FS-MTL)
xm
xn
LSTM
LSTM
LSTM
softmax
softmax
Lmtask
Lntask
(b) Shared-Private Model (SP-MTL)
Figure 2: Two architectures for learning multiple tasks. Yellow and gray boxes represent shared and private LSTM layers respectively.
classification by learning tasks in parallel. To facilitate this, we give some explanation for notations used in this paper. Formally, we refer to Dk as a dataset with Nk samples for task k. Specifically,
Dk = {(xki , yki )} Nk i=1 (7)
where xki and y k i denote a sentence and corresponding label for task k.
3.1 Two Sharing Schemes for Sentence Modeling
The key factor of multi-task learning is the sharing scheme in latent feature space. In neural network based model, the latent features can be regarded as the states of hidden neurons. Specific to text classification, the latent features are the hidden states of LSTM at the end of a sentence. Therefore, the sharing schemes are different in how to group the shared features. Here, we first introduce two sharing schemes with multi-task learning: fully-shared scheme and shared-private scheme.
Fully-SharedModel (FS-MTL) In fully-shared model, we use a single shared LSTM layer to extract features for all the tasks. For example, given two tasks m and n, it takes the view that the features of taskm can be totally shared by task n and vice versa. This model ignores the fact that some features are task-dependent. Figure 2a illustrates the fully-shared model.
Shared-Private Model (SP-MTL) As shown in Figure 2b, the shared-private model introduces two feature spaces for each task: one is used to store task-dependent features, the other is used
to capture task-invariant features. Accordingly, we can see each task is assigned a private LSTM layer and shared LSTM layer. Formally, for any sentence in task k, we can compute its shared representation skt and task-specific representation h k t as follows:
skt = LSTM(xt, s k t−1, θs), (8)
hkt = LSTM(xt,h m t−1, θk) (9)
where LSTM(., θ) is defined as Eq. (4). The final features are concatenation of the features from private space and shared space.
3.2 Task-Specific Output Layer
For a sentence in task k, its feature h(k), emitted by the deep muti-task architectures, is ultimately fed into the corresponding task-specific softmax layer for classification or other tasks. The parameters of the network are trained to minimise the cross-entropy of the predicted and true distributions on all the tasks. The loss Ltask can be computed as:
LTask = K∑ k=1 αkL(ŷ (k), y(k)) (10)
where αk is the weights for each task k respectively. L(ŷ, y) is defined as Eq. 6.
4 Incorporating Adversarial Training
Although the shared-private model separates the feature space into the shared and private spaces, there is no guarantee that sharable features can not exist in private feature space, or vice versa. Thus, some useful sharable features could be ignored in shared-private model, and the shared feature space is also vulnerable to contamination by some taskspecific information. Therefore, a simple principle can be applied into multi-task learning that a good shared feature space should contain more common information and no task-specific information. To address this problem, we introduce adversarial training into multi-task framework as shown in Figure 3 (ASPMTL).
4.1 Adversarial Network
Adversarial networks have recently surfaced and are first used for generative model (Goodfellow
4
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
xm
xn
LSTM
LSTM
LSTM
LDiff LAdvLDiff
softmax
softmax
Lmtask
Lntask
Figure 3: Adversarial shared-private model. Yellow and gray boxes represent shared and private LSTM layers respectively.
et al., 2014). The goal is to learn a generative distribution pG(x) that matches the real data distribution Pdata(x) Specifically, GAN learns a generative network G and discriminative model D, in which G generates samples from the generator distribution pG(x). and D learns to determine whether a sample is from pG(x) or Pdata(x). This min-max game can be optimized by the following risk:
ϕ = min G max D
( Ex∼Pdata [logD(x)]
+ Ez∼p(z)[log(1−D(G(z)))] )
(11)
While originally proposed for generating random samples, adversarial network can be used as a general tool to measure equivalence between distributions (Taigman et al., 2016). Formally, (Ajakan et al., 2014) linked the adversarial loss to the H-divergence between two distributions and successfully achieve unsupervised domain adaptation with adversarial network. Motivated by theory on domain adaptation (Ben-David et al., 2010, 2007; Bousmalis et al., 2016) that a transferable feature is one for which an algorithm cannot learn to identify the domain of origin of the input observation.
4.2 Task Adversarial Loss for MTL
Inspired by adversarial networks (Goodfellow et al., 2014), we proposed an adversarial sharedprivate model for multi-task learning, in which a shared recurrent neural layer is working adversarially towards a learnable multi-layer perceptron, preventing it from making an accurate prediction about the types of tasks. This adversarial training encourages shared space to be more pure and ensure the shared representation not be contaminated by task-specific features.
Task Discriminator Discriminator is used to map the shared representation of sentences into a
probability distribution, estimating what kinds of tasks the encoded sentence comes from.
D(skT , θD) = softmax(b+Us k T ) (12)
whereU ∈ Rd×d is a learnable parameter and b ∈ Rd is a bias.
Adversarial Loss Different with most existing multi-task learning algorithm, we add an extra task adversarial loss LAdv to prevent task-specific feature from creeping in to shared space. The task adversarial loss is used to train a model to produce shared features such that a classifier cannot reliably predict the task based on these features. The original loss of adversarial network is limited since it can only be used in binary situation. To overcome this, we extend it to multi-class form, which allow our model can be trained together with multiple tasks:
LAdv = min θs
( λmax
θD ( K∑ k=1 Nk∑ i=1 dki log[D(E(x k))])
) (13)
where dki denotes the ground-truth label indicating the type of the current task. Here, there is a minmax optimization and the basic idea is that, given a sentence, the shared LSTM generates a representation to mislead the task discriminator. At the same time, the discriminator tries its best to make a correct classification on the type of task. After the training phase, the shared feature extractor and task discriminator reach a point at which both cannot improve and the discriminator is unable to differentiate among all the tasks.
Semi-supervised Learning Multi-task Learning We notice that the LAdv requires only the input sentence x and does not require the corresponding label y, which makes it possible to combine our model with semi-supervised learning. Finally, in this semi-supervised multi-task learning framework, our model can not only utilize the data from related tasks, but can employ abundant unlabeled corpora.
4.3 Orthogonality Constraints
We notice that there is a potential drawback of the above model. That is, the task-invariant features can appear both in shared space and private space. Motivated by recently work(Jia et al., 2010; Salzmann et al., 2010; Bousmalis et al., 2016)
5
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
Dataset Train Dev. Test Unlab. Avg. L Vocab.
Books 1400 200 400 2000 159 62K Elec. 1398 200 400 2000 101 30K DVD 1400 200 400 2000 173 69K Kitchen 1400 200 400 2000 89 28K Apparel 1400 200 400 2000 57 21K Camera 1397 200 400 2000 130 26K Health 1400 200 400 2000 81 26K Music 1400 200 400 2000 136 60K Toys 1400 200 400 2000 90 28K Video 1400 200 400 2000 156 57K Baby 1300 200 400 2000 104 26K Mag. 1370 200 400 2000 117 30K Soft. 1315 200 400 2000 129 26K Sports 1400 200 400 2000 94 30K IMDB 1400 200 400 2000 269 44K MR 1400 200 400 2000 21 12K
Table 1: Statistics of the 16 datasets. The columns 2-5 denote the number of samples in training, development, test and unlabeled sets. The last two columns represent the average length and vocabulary size of corresponding dataset.
on shared-private latent space analysis, we introduce orthogonality constraints, which penalize redundant latent representations and encourages the shared and private extractors to encode different aspects of the inputs.
After exploring many optional methods, we find below loss is optimal, which is used by Bousmalis et al. (2016) and achieve a better performance:
Ldiff = K∑ k=1 ∥∥∥Sk⊤Hk∥∥∥2 F , (14)
where ∥ · ∥2F is the squared Frobenius norm. S k and Hk are two matrics, whose rows are the output of shared extractor Es(, ; θs) and task-specific extrator Ek(, ; θk) of a input sentence.
4.4 Put It All Together
The final loss function of our model can be written as:
L = LTask + λLAdv + γLDiff (15)
where λ and γ are hyper-parameter. The networks are trained with backpropagation and this minimax optimization becomes possible via the use of a gradient reversal layer (Ganin and Lempitsky, 2015).
5 Experiment
5.1 Dataset
To make an extensive evaluation, we collect 16 different datasets from several popular review corpora. The first 14 datasets are product reviews, which contain Amazon product reviews from different domains, such as Books, DVDs, Electronics, ect. The goal is to classify a product review as either positive or negative. These datasets are collected based on the raw data 1 provided by (Blitzer et al., 2007). Specifically, we extract the sentences and corresponding labels from the unprocessed original data 2. The only preprocessing operation of these sentences is tokenized using the Stanford tokenizer 3. The remaining two datasets are about movie reviews. The IMDB dataset4 consists of movie reviews with binary classes (Maas et al., 2011). One key aspect of this dataset is that each movie review has several sentences. The MR dataset also consists of movie reviews from rotten tomato website with two classes 5(Pang and Lee, 2005). All the datasets in each task are partitioned randomly into training set, development set and testing set with the proportion of 70%, 20% and 10% respectively. The detailed statistics about all the datasets are listed in Table 1.
5.2 Competitor Methods for Multi-task Learning
The multi-task frameworks proposed by previous works are various while not all can be applied to the tasks we focused. Nevertheless, we chose two most related neural models for multi-task learning and implement them as competitor methods.
• MT-CNN: This model is proposed by Collobert and Weston (2008) with convolutional layer, in which lookup-tables are shared partially while other layers are task-specific.
1https://www.cs.jhu.edu/˜mdredze/ datasets/sentiment/
2Blitzer et al. (2007) also provides two extra processed datasets with the format of Bag-of-Words, which are not proper for neural-based models.
3http://nlp.stanford.edu/software/ tokenizer.shtml
4https://www.cs.jhu.edu/˜mdredze/ datasets/sentiment/unprocessed.tar.gz
5https://www.cs.cornell.edu/people/ pabo/movie-review-data/.
6
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
Task Single Task Multiple Tasks
LSTM BiLSTM sLSTM Avg. MT-DNN MT-CNN FS-MTL SP-MTL ASP-MTL
Books 20.5 19.0 18.0 19.2 17.7(−1.5) 15.6(−3.6) 17.5(−1.7) 18.7(−0.5) 13.0(−6.2) Electronics 19.5 21.5 23.3 21.4 18.2(−3.2) 16.9(−4.5) 14.3(−7.1) 12.3(−9.1) 11.0(−10.4) DVD 18.3 19.5 22.0 19.9 15.8(−4.1) 16.1(−3.8) 16.5(−3.4) 16.1(−3.8) 12.6(−7.3) Kitchen 22.0 18.8 19.5 20.1 19.2(−0.9) 16.8(−3.3) 14.0(−6.1) 14.8(−5.3) 12.8(−7.3) Apparel 16.8 14.0 16.3 15.7 14.9(−0.8) 16.1(+0.4) 15.5(−0.2) 13.4(−2.3) 11.3(−4.4) Camera 14.8 14.0 15.0 14.6 13.7(−0.9) 14.0(−0.6) 13.5(−1.1) 12.1(−2.5) 8.7(−5.9) Health 15.5 21.3 16.5 17.8 14.3(−3.5) 12.9(−4.9) 12.0(−5.8) 12.8(−5.0) 10.9(−6.9) Music 23.3 22.8 23.0 23.0 15.3(−7.7) 16.3(−6.7) 18.8(−4.2) 17.0(−6.0) 17.4(−5.6) Toys 16.8 15.3 16.8 16.3 12.1(−4.2) 10.9(−5.4) 15.5(−0.8) 14.9(−1.4) 11.2(−5.1) Video 18.5 16.3 16.3 17.0 15.0(−2.0) 18.7(+1.7) 16.3(−0.7) 16.8(−0.2) 14.5(−2.5) Baby 15.3 16.5 15.8 15.9 12.1(−3.8) 12.4(−3.5) 12.0(−3.9) 13.2(−2.7) 10.2(−5.7) Magazines 10.8 8.5 12.3 10.5 10.6(+0.1) 12.3(+1.8) 7.5(−3.0) 8.1(−2.4) 7.6(−2.9) Software 15.3 14.3 14.5 14.7 14.4(−0.3) 13.4(−1.3) 13.8(−0.9) 13.1(−1.6) 12.7(−2.0) Sports 18.3 16.0 17.5 17.3 16.8(−0.5) 16.1(−1.2) 14.5(−2.8) 12.7(−4.6) 13.3(−4.0) IMDB 18.3 15.0 18.5 17.3 16.7(−0.6) 13.7(−3.6) 17.5(+0.2) 15.2(−2.1) 14.2(−3.1) MR 27.3 25.3 28.0 26.9 24.5(−2.4) 25.5(−1.4) 25.3(−1.6) 24.1(−2.8) 22.7(−4.2)
AVG 18.2 17.4 18.3 18.0 15.7(−2.3) 15.5(−2.5) 15.3(−2.7) 14.7(−3.3) 12.8(−5.2)
Table 2: Error rates of our models on 16 datasets against typical baselines. The numbers in brackets represent the improvements relative to the average performance (Avg.) of three single task baselines.
• MT-DNN: The model is proposed by Liu et al. (2015) with bag-of-words input and multi-layer perceptrons, in which a hidden layer is shared.
5.3 Hyperparameters
The word embeddings for all of the models are initialized with the 200d GloVe vectors (840B token version, (Pennington et al., 2014)). The other parameters are initialized by randomly sampling from uniform distribution in [−0.1, 0.1]. The minibatch size is set to 16.
For each task, we take the hyperparameters which achieve the best performance on the development set via an small grid search over combinations of the initial learning rate [0.1, 0.01], λ ∈ [0.01, 0.1], and γ ∈ [0.01, 0.1]. Finally, we chose the learning rate as 0.01, λ as 0.05 and γ as 0.01.
5.4 Performance Evaluation
Table 2 shows the error rates on 16 text classification tasks. The column of “Single Task” shows the results of vanilla LSTM, bidirectional LSTM (BiLSTM), stacked LSTM (sLSTM) and the average error rates of previous three models. The column of “Multiple Tasks” shows the results achieved by corresponding multi-task models. From this table, we can see that the performance of most tasks can be improved with a large margin with the help of multi-task learning, in which our model achieves the lowest error rates.
More concretely, compared with SP-MTL, ASPMTL achieves 5.2% average improvement surpassing SP-MTL with 1.9%, which indicates the importance of adversarial learning. It is noteworthy that for FS-MTL, the performances of some tasks are degraded, since this model puts all private and shared information into a unified space.
5.5 Shared Knowledge Transfer
With the help of adversarial learning, the shared feature extractor Es can generate more pure taskinvariant representations, which can be considered as off-the-shelf knowledge and then be used for unseen new tasks. To test the transferability of our learned shared extractor, we also design an experiment, in which we take turns choosing 15 tasks to train our model MS with multi-task learning, then the learned shared layer are transferred to a second network MT that is used for the remaining one task. The parameters of transferred layer are kept frozen, and the rest of parameters of the network MT are randomly initialized. More formally, we investigate two mechanisms towards the transferred shared extractor. As shown in Figure 4. The first one Single Channel (SC) model consists of one shared feature extractor Es from MS , then the extracted representation will be sent to an output layer. By contrast, the BiChannel (BC) model introduces an extra LSTM layer to encode more task-specific information. To evaluate the effectiveness of our introduced adver-
7
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
Source Tasks Single Task Transfer Models
LSTM BiLSTM sLSTM Avg. SP-MTL-SC SP-MTL-BC ASP-MTL-SC ASP-MTL-BC
ϕ (Books) 20.5 19.0 18.0 19.2 17.8(−3.6) 16.2(−3.0) 16.7(−2.5) 13.3(−5.9) ϕ (Electronics) 19.5 21.5 23.3 21.4 15.1(−4.5) 14.6(−6.8) 15.1(−6.3) 14.9(−6.5) ϕ (DVD) 18.3 19.5 22.0 19.9 14.7(−3.8) 15.5(−4.4) 12.1(−7.8) 12.4(−7.5) ϕ (Kitchen) 22.0 18.8 19.5 20.1 15.0(−3.3) 16.6(−3.5) 14.6(−5.5) 14.1(−6.0) ϕ (Apparel) 16.8 14.0 16.3 15.7 14.9(+0.4) 12.3(−3.4) 11.6(−4.1) 13.6(−2.1) ϕ (Camera) 14.8 14.0 15.0 14.6 13.1(−0.6) 12.1(−2.5) 11.6(−3.0) 10.3(−4.3) ϕ (Health) 15.5 21.3 16.5 17.8 14.1(−4.9) 14.2(−3.6) 12.2(−5.6) 10.5(−7.3) ϕ (Music) 23.3 22.8 23.0 23.0 19.9(−6.7) 17.9(−5.1) 16.4(−6.6) 18.2(−4.8) ϕ (Toys) 16.8 15.3 16.8 16.3 13.8(−5.4) 12.2(−4.1) 13.0(−4.7) 11.2(−5.1) ϕ (Video) 18.5 16.3 16.3 17.0 14.2(+1.7) 15.1(−1.9) 14.8(−2.2) 14.8(−2.2) ϕ (Baby) 15.3 16.5 15.8 15.9 16.6(−3.5) 16.9(+1.0) 11.5(−4.4) 10.0(−5.9) ϕ (Magazines) 10.8 8.5 12.3 10.5 10.6(+1.8) 10.2(−0.3) 8.6(−1.9) 9.7(−0.8) ϕ (Software) 15.3 14.3 14.5 14.7 13.0(−1.3) 12.7(−2.0) 14.3(−0.4) 11.1(−3.6) ϕ (Sports) 18.3 16.0 17.5 17.3 16.3(−1.2) 16.2(−1.1) 13.4(−3.9) 13.6(−3.7) ϕ (IMDB) 18.3 15.0 18.5 17.3 12.4(−3.6) 12.8(−4.5) 12.5(−4.8) 13.3(−4.0) ϕ (MR) 27.3 25.3 28.0 26.9 26.0(−1.4) 26.5(−0.4) 22.7(−4.2) 23.5(−3.4)
AVG 18.2 17.4 18.3 18.0 15.5(−2.5) 15.1(−2.9) 13.6(−4.2) 13.4(−4.6)
Table 3: Error rates of our models on 16 datasets against vanilla multi-task learning. ϕ (Books) means that we transfer the knowledge of the other 15 tasks to the target task Books.
xt LSTM softmax
Es
(a) Single Channel
xt LSTM
LSTM
softmax
Es
(b) Bi-Channel
Figure 4: Two transfer strategies using a pretrained shared LSTM layer. Yellow box denotes shared feature extractor Es trained by 15 tasks.
sarial training framework, we also make a comparison with vanilla multi-task learning method.
Results and Analysis As shown in Table 3, we can see the shared layer from ASP-MTL achieves a better performance compared with SP-MTL. Besides, for the two kinds of transfer strategies, the Bi-Channel model performs better. The reason is that the task-specific layer introduced in the BiChannel model can store some private features. Overall, the results indicate that we can save the existing knowledge into a shared recurrent layer using adversarial multi-task learning, which is quite useful for a new task.
5.6 Visualization
To get an intuitive understanding of how the introduced orthogonality constraints worked compared with vanilla shared-private model, we design an experiment to examine the behaviors of neurons from private layer and shared layer. More concretely, we refer to htj as the activation of the j-
neuron at time step t, where t ∈ {1, . . . , n} and j ∈ {1, . . . , d}. By visualizing the hidden state hj and analyzing the maximum activation, we can find what kinds of patterns the current neuron focuses on. Figure 5 illustrates this phenomenon. Here, we randomly sample a sentence from the validation set of Baby task and analyze the changes of the predicted sentiment score at different time steps, which are obtained by SP-MTL and our proposed model. Additionally, to get more insights into how neurons in shared layer behave diversely towards different input word, we visualize the activation of two typical neurons. For the positive sentence “Five stars, my baby can fall asleep soon in the stroller”, both models capture the informative pattern “Five stars” 6. However, SP-MTL makes a wrong prediction due to misunderstanding of the word “asleep”. By contrast, our model makes a correct prediction and the reason can be inferred from the activation of Figure 5-(b), where the shared layer of SP-MTL is so sensitive that many features related to other tasks are included, such as ”asleep”, which misleads the final prediction. This indicates the importance of introducing adversarial learning to prevent the shared layer from being contaminated by task-specific features.
6For this case, the vanilla LSTM also give a wrong answer due to ignoring the feature “Five stars”.
8
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
Five stars , my baby can fall asleep soon in the stroller 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 SP-MTL Ours
(a) Predicted Sentiment Score by Two Models (b) Behaviours of Neuron hs18 and h s 21
Figure 5: (a) The change of the predicted sentiment score at different time steps. Y-axis represents the sentiment score, while X-axis represents the input words in chronological order. The darker grey horizontal line gives a border between the positive and negative sentiments. (b) The blue heat map describes the behaviour of neuron hs18 from shared layer of SP-MTL, while the purple one is used to show the behaviour of neuron hs21, which belongs to the shared layer of our model.
Model Shared Layer Task-Movie Task-Baby
SP-MTL
good, great bad, love, simple, cut, slow, cheap, infantile good, great, well-directed, pointless, cut, cheap, infantile love, bad, cute, safety, mild, broken simple
ASP-MTL good, great, love, bad poor well-directed, pointless, cut, cheap, infantile cute, safety, mild, broken simple
Table 4: Typical patterns captured by shared layer and task-specific layer of SP-MTL and ASP-MTL models on Movie and Baby tasks.
We also list some typical patterns captured by neurons from shared layer and task-specific layer in Table 4, and we have observed that: 1) for SP-MTL, if some patterns are captured by taskspecific layer, they are likely to be placed into shared space. Clearly, suppose we have many tasks to be trained jointly, the shared layer bear much pressure and must sacrifice substantial amount of capacity to capture the patterns they actually do not need. Furthermore, some typical taskinvariant features also go into task-specific layer. 2) for ASP-MTL, we find the features captured by shared and task-specific layer have a small amount of intersection, which allows these two kinds of layers can work effectively.
6 Related Work
There are two threads of related work. One thread is multi-task learning with neural network. Neural networks based multi-task learning has been proven effective in many NLP problems (Collobert and Weston, 2008; Glorot et al., 2011; Liu et al., 2015, 2016). In most of these models, the
lower layers are shared across all tasks, while top layers are task-specific. These work has potential limitation of just learning a shared space solely on sharing parameters, while our model introduce two strategies to learn the clear and non-redundant shared-private space. Another thread of work is adversarial network. Adversarial networks have recently surfaced as a general tool measure equivalence between distributions and it has proven to be effective in a variety of tasks. Ajakan et al. (2014); Bousmalis et al. (2016) applied adverarial training to domain adaptation, aiming at transferring the knowledge of one source domain to target domain. Park and Im (2016) proposed a novel approach for multimodal representation learning which uses adversarial back-propagation concept. Different from these models, our model aims to find task-invariant sharable information for multiple related tasks using adversarial training strategy. Moreover, we extend binary adversarial training to multi-class, which enable multiple tasks to be jointly trained.
7 Conclusion
In this paper, we have proposed an adversarial multi-task learning framework, in which the taskspecific and task-invariant features are learned non-redundantly, therefore capturing the sharedprivate separation of different tasks. We have demonstrated the effectiveness of our approach by applying our model to 16 different text classification tasks. We also perform extensive qualitative analysis, deriving insights and indirectly explaining the quantitative improvements in the overall performance.
9
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899

This paper introduces new configurations and training objectives for neural sequence models in a multi-task setting. As the authors describe well, the multi-task setting is important because some tasks have shared information and in some scenarios learning many tasks can improve overall performance.  The methods section is relatively clear and logical, and I like where it ended up, though it could be slightly better organized. The organization that I realized after reading is that there are two problems: 1) shared features end up in the private feature space, and 2) private features end up in the  shared space. There is one novel method for each problem. That organization up front would make the methods more cohesive. In any case, they introduce one  method that keeps task-specific features out of shared representation (adversarial loss) and another to keep shared features out of task-specific representations (orthogonality constraints). My only point of confusion is the adversarial system. After LSTM output there is another layer, D(s^k_T,   theta_D), relying on parameters U and b. This output is considered a probability distribution which is compared against the actual. This means it is possible it will just learn U and b that effectively mask task-specific information from  the LSTM outputs, and doesn't  seem like it can guarantee task-specific information is removed.  Before I read the evaluation section I wrote down what I hoped the experiments would look like and it did most of it. This is an interesting idea and there are  a lot more experiments one can imagine but I think here they have the basics to show the validity of their methods. It would be helpful to have best known results on these tasks.  My primary concern with this paper is the lack of deeper motivation for the  approach. I think it is easy to understand that in a totally shared model there will be problems due to conflicts in feature space. The extension to  partially shared features seems like a reaction to that issue -- one would  expect that the useful shared information is in the shared latent space and  each task-specific space would learn features for that space. Maybe this works and maybe it doesn't, but the logic is clear to me. In contrast, the authors seem to start from the assumption that this  shared-private  model has this issue. I expected the argument flow to be 1) Fully-shared obviously has this problem; 2) shared-private seems to address this; 3) in practice shared-private does not fully address this issue for reasons a,b,c.; 4) we introduce a method that more effectively constrains the spaces. Table 4 helped me to partially understand what's going wrong with shared-private and what your methods do; some terms are _usually_ one connotation or another, and that general trend can probably get them into the shared feature space. This simple explanation, an example, and a more logical argument flow would help the introduction and make this a really nice reading paper.  Finally, I think this research ties into some other uncited MTL work [1], which does deep hierarchical MTL - supervised POS tagging at a lower level, chunking at the next level up, ccg tagging higher, etc. They then discuss at the end some of the qualities that make MTL possible and conclude that MTL only works  when tasks are sufficiently similar.  The ASP-MTL paper made me think of this previous work because potentially this model could learn what sufficiently similar is -- i.e., if two tasks are not sufficiently similar the shared model would learn nothing and it would fall back to learning two independent systems, as compared to a shared-private model baseline that might overfit and perform poorly.  [1] @inproceedings{sogaard2016deep,   title={Deep multi-task learning with low level tasks supervised at lower layers},   author={S{  o}gaard, Anders and Goldberg, Yoav},   booktitle={Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics},   volume={2},   pages={231--235},   year={2016},   organization={Association for Computational Linguistics} }


Neural End-to-End Learning for Computational Argumentation Mining
We investigate neural techniques for endto-end computational argumentation mining. We frame the problem as a tokenbased dependency parsing as well as a token-based sequence tagging model, including a multi-task learning setup. Contrary to models that operate on the argument component level, we find that framing the problem as dependency parsing leads to subpar performance results. In contrast, less complex (local) tagging models based on BiLSTMs perform robustly across classification scenarios, being able to catch long-range dependencies inherent to the argumentation mining problem. Moreover, we find that jointly learning ‘natural’ subtasks, in a multi-task learning setup, improves performance.
1 Introduction
Computational argumentation mining (AM) deals with finding argumentation structures in text. This involves several subtasks, such as: (a) separating argumentative units from non-argumentative units, also called ‘component segmentation’; (b) classifying argument components into classes such as “Premise” or “Claim”; (c) finding relations between argument components; (d) classifying relations into classes such as “Support” or “Attack” (Persing and Ng, 2016; Stab and Gurevych, 2016).
Thus, AM would have to detect claims and premises (reasons) in texts such as the following, where premise P supports claim C:
Since it killed many marine livesP , ::::::: tourism ::: has :::::::::: threatened :::::: natureC .
Argument structures in real texts are typically much more complex, cf. Figure 1.
While different research has addressed different subsets of the AM problem (see below), the ultimate goal is to solve all of them, starting from unannotated plain text. Two recent approaches to this end-to-end learning scenario are Persing and Ng (2016) and Stab and Gurevych (2016). Both solve the end-to-end task by first training independent models for each subtask and then defining an integer linear programming (ILP) model that encodes global constraints such as that each premise has a parent, etc. Besides their pipeline architecture the approaches also have in common that they heavily rely on hand-crafted features.
Hand-crafted features pose a problem because AM is to some degree an “arbitrary” problem in that the notion of “argument” critically relies on the underlying argumentation theory (Reed et al., 2008; Biran and Rambow, 2011; Habernal and Gurevych, 2015; Stab and Gurevych, 2016). Accordingly, datasets typically differ with respect to their annotation of (often rather complex) argument structure. Thus, feature sets would have to be manually adapted to and designed for each new sample of data, a challenging task. The same critique applies to the designing of ILP constraints. Moreover, from a machine learning perspective, pipeline approaches are problematic because they solve subtasks independently and thus lead to error propagation rather than exploiting interrelationships between variables. In contrast to this, we investigate neural techniques for end-to-end learning in computational AM, which do not require the hand-crafting of features or constraints. The models we survey also all capture some notion of “joint”—rather than “pipeline”—learning. We investigate several approaches.
First, we frame the end-to-end AM problem as a dependency parsing problem. Dependency parsing may be considered a natural choice for AM, because argument structures often form trees, or
2
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
closely resemble them (see §3). Hence, it is not surprising that “discourse parsing” has been suggested for AM (Peldszus and Stede, 2015). What distinguishes our approach from these previous ones is that we operate on the token level, rather than on the level of components, because we address the end-to-end framework and, thus, do not assume that non-argumentative units have already been sorted out and/or that the boundaries of argumentative units are given.
Second, we frame the problem as a sequence tagging problem. This is a “natural” choice especially for component segmentation and typing, which is a typical entity recognition problem for which BIO tagging is a standard approach, pursued in AM, e.g., by Habernal and Gurevych (2016). The challenge in the end-to-end setting is to also include relations into the tagging scheme, which we realize by coding the distances between linked components into the tag label. Since related entities in AM are oftentimes several dozens of tokens apart from each other, neural sequence tagging models are in principle ideal candidates for such a framing because they can take into account long-range dependencies—something that is inherently difficult to capture with traditional feature-based tagging models such as conditional random fields (CRFs).
Third, we frame AM as a multi-task (tagging) problem (Caruana, 1997; Collobert and Weston, 2008). We experiment with subtasks of AM— e.g., component detection—as auxiliary tasks and investigate whether this improves performance on the AM problem. Adding such subtasks can be seen as analogous to de-coupling, e.g., component detection from the full AM problem.
Fourth, we evaluate the model of Miwa and Bansal (2016) that combines sequential (entity) and tree structure (relation) information and is in principle applicable to any problem where the aim is to extract entities and their relations. As such, this model makes fewer assumptions than our dependency parsing and tagging approaches.
The contributions of this paper are as follows. (1) We present the first neural end-to-end solutions to computational AM. (2) We show that several of them perform better than the state-of-theart joint ILP model. (3) We show that a framing of AM as a token-based dependency parsing problem is ineffective—in contrast to what has been proposed for systems that operate on the coarser
component level and that (4) a standard neural sequence tagging model that encodes distance information between components performs robustly in different environments. Finally, (5) we show that a multi-task learning setup where natural subtasks of the full AM problem are added as auxiliary tasks improves performance.
2 Related Work
AM has applications in legal decision making (Palau and Moens, 2009; Moens et al., 2007), document summarization, and the analysis of scientific papers (Kirschner et al., 2015). Its importance for the educational domain has been highlighted by recent work on writing assistance (Zhang and Litman, 2016) and essay scoring (Persing and Ng, 2015; Somasundaran et al., 2016).
Most works on AM address subtasks of AM such as locating/typing components (Florou et al., 2013; Moens et al., 2007; Rooney et al., 2012; Knight et al., 2003; Levy et al., 2014; Rinott et al., 2015). Relatively few works address the full AM problem of component and relation identification. Peldszus and Stede (2016) present a corpus of microtexts containing only argumentatively relevant text of controlled complexity. To our best knowledge, Stab and Gurevych (2016) created the only corpus of attested high quality which annotates the AM problem in its entire complexity: it contains token-level annotations of components, their types, as well as relations and their types.
3 Data
We use the dataset of persuasive essays (PE) from Stab and Gurevych (2016), which contains student essays written in response to controversial topics such as “competition or cooperation—which is better?”
The corpus consists of 402 essays, 80 of which are reserved for testing. The annotation distinguishes between major claims (the central position of an author with respect to the essay’s topic), claims (controversial statements that are either for or against the major claims), and premises, which give reasons for claims or other premises and either support or attack them. Overall, there are 751 major claims, 1506 claims, and 3832 premises.
The corpus has a special structure, illustrated in Figure 1. First, major claims relate to no other components. Second, claims always relate to all other major claims. Third, each premise relates
3
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
MC1 MC2
C1 C2 C3
P1 P2 P3 P4 P5 P6
MC1 C1 P1 P2 P3 P4 C2
P5 P6 C3 MC2
Figure 1: Bottom: Linear argumentation structure in a student essay. The essay is comprised of nonargumentative units (square blue) and argumentative units of different types: Premises (P), claims (C) and major claims (MC). Top: Relationsships between argumentative units. Black arrows are support (for), red dashed arrows are attack (against).
to exactly one claim or premise. Thus, the argument structure in each essay is—almost—a tree. Since there may be several major claims, each claim connects to multiple targets, which violates the tree structure. This does not pose a problem, however, since we can “loss-lessly” re-link the claims to one of the major claims (e.g., the last major claim in a document) and create a special root node to which the major claims link.
There is another peculiarity of this data. Each essay is divided into paragraphs, of which there are 2235 in total (1786 train, 449 test). The argumentation structure is completely contained within a paragraph, except, possibly, for the relation from claims to major claims. Paragraphs have an average length of 66 tokens and are therefore much shorter than essays, which have an average length of 368 tokens. Thus, prediction on the paragraph level is easier than prediction on the essay level, because there are fewer components in a paragraph and hence fewer possibilities of source and target components in argument relations. Paragraphs are not only easier for relation classification, but also for component classification: a paragraph can never contain premises only, for example, since premises link to other components.
4 Models
This section describes our neural network framings for end-to-end AM.
Sequence Tagging is the problem of assigning each element in a stream of input tokens a
label. In a neural context, the natural choice for tagging problems are recurrent neural nets (RNNs) in which a hidden vector representation ht at time point t depends on the previous hidden vector representation ht−1 and the input xt. In this way, an infinite window (“long-range dependencies”) around the current input token xt can be taken into account when making an output prediction yt. We choose particular RNNs, namely, LSTMs (Hochreiter and Schmidhuber, 1997), which are popular for being able to address vanishing/exploding gradients problems. In addition to considering a left-to-right flow of information, bidirectional LSTMs also capture information to the right of the current input token.
The most recent generation of neural tagging models add label dependencies to BiLSTMs (BL), so that successive output decisions are not made independently. This class of models is called BiLSTM-CRF (BLC) (Huang et al., 2015). The model of Ma and Hovy (2016) adds convolutional neural nets (CNNs) on the character-level to BiLSTM-CRFs, leading to BiLSTM-CRF-CNN (BLCC) models. The character-level CNN may address problems of out-of-vocabulary words, that is, words not seen during training.
AM as Sequence Tagging: We frame AM as the following sequence tagging problem. Each input token has an associated label from Y , where
Y = {(b, t, d, s) | b ∈ {B, I,O}, t ∈ {P,C,MC,⊥}, d ∈ {. . . ,−2,−1, 1, 2, . . . ,⊥}, s ∈ {Supp,Att, For,Ag,⊥}}. (1)
4
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
In other words, Y consists of all four-tuples (b, t, d, s) where b is a BIO encoding indicating whether the current token is non-argumentative (O) or begins (B) or continues (I) a component; t indicates the type of the component (claim C, premise P, or major claim MC for our data). Moreover, d encodes the distance—measured in number of components—between the current component and the component it relates to. We encode the same d value for each token in a given component. Finally, s is the relation type (“stance”) between two components and its value may be Support (Supp), Attack (Att), or For or Against (Ag). We also have a special symbol ⊥ that indicates when a particular slot is not filled: e.g., a nonargumentative unit (b = O) has neither component type, nor relation, nor relation type. We refer to this framing as STagT (for “Simple Tagging”), where T refers to the tagger used. For the example from §1, our coding would hence be:
Since it killed many (O,⊥,⊥,⊥) (B,P,1,Supp) (I,P,1,Supp) (I,P,1,Supp)
marine lives , tourism (I,P,1,Supp) (I,P,1,Supp) (O,⊥,⊥,⊥) (B,C,⊥,For)
has threatened nature . (I,C,⊥,For) (I,C,⊥,For) (I,C,⊥,For) (O,⊥, ⊥, ⊥)
While the size of the label set Y is potentially infinite, we would expect it to be finite even in a potentially infinitely large data set, because humans also have only finite memory and are therefore expected to keep related components close in textual space. Indeed, as Figure 2 shows, in our PE essay data set about 30% of all relations between components have distance −1, that is, they follow the claim or premise that they attach to. Overall, around 2/3 of all relation distances d lie in {−2,−1, 1}. However, the figure also illustrates that there are indeed long-range dependencies: distance values between −11 and +10 are observed in the data.
Dependency Parsing methods can be classified into graph-based and transition-based approaches (Kiperwasser and Goldberg, 2016). Transitionbased parsers encode the parsing problem as a sequence of configurations which may be modified by application of actions such as shift, reduce, etc. The system starts with an initial configuration in which sentence elements are on a buffer and a stack, and a classifier successively decides which action to take next, leading to different configura-
0
5
10
15
20
25
30
−10 −5 0 5 10
%
d
d
Figure 2: Distribution of distances d between components in PE dataset.
tions. The system terminates after a finite number of actions, and the parse tree is read off the terminal configuration. Graph-based parsers solve a structured prediction problem in which the goal is learning a scoring function over dependency trees such that correct trees are scored above all others.
Traditional dependency parsers used handcrafted feature functions that look at “core” elements such as “word on top of the stack”, “POS of word on top of the stack”, and conjunctions of core features such as “word is X and POS is Y” (see McDonald et al. (2005)). Most neural parsers have not entirely abandoned feature engineering. Instead, they rely, for example, on encoding the core features of parsers as low-dimensional embedding vectors (Chen and Manning, 2014) but ignore feature combinations. Kiperwasser and Goldberg (2016) design a neural parser that uses only four features: the BiLSTM vector representations of the top 3 items on the stack and the first item on the buffer. In contrast, Dyer et al. (2015)’s neural parser associates each stack with a “stack LSTM” that encodes their contents. Actions are chosen based on the stack LSTM representations of the stacks, and no more feature engineering is necessary. Moreover, their parser has thus access to any part of the input, its history and stack contents.
AM as Dependency Parsing: To frame a problem as a dependency parsing problem, each instance of the problem must be encoded as a directed tree, where tokens have heads, which in turn are labeled. For end-to-end AM, we propose the framing illustrated in Figure 3. We highlight two design decisions, the remaining are analogous and/or can be read off the figure.
• The head of each non-argumentative text token is the document terminating token END, which is a punctuation mark in all our cases. The label of this link is O, the symbol for
5
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
non-argumentative units.
• The head of each token in a premise is the first token of the claim or premise that it links to. The label of each of these links is (b,P,Supp) or (b,P,Att) depending on whether a premise “supports” or “attacks” a claim or premise; b ∈ {B, I}.
1 2 3 4 5 6 7 8 9 10 11 12
O
(B,P,Supp) (I,P,Supp)
O
(B,C,For)
Figure 3: Dependency representation of sample sentence from §1. Links and selected labels.
Multi-Task Learning Recently, there has been a lot of interest in so-called multi-task learning (MTL) scenarios, where several tasks are learned jointly (Søgaard and Goldberg, 2016; Peng and Dredze, 2016; Yang et al., 2016; Rusu et al., 2016; Héctor and Plank, 2017). It has been argued that such learning scenarios are closer to human learning because humans often transfer knowledge between several domains. In a neural context, MTL is often implemented via weight sharing: several tasks are trained in the same network architecture, thereby sharing a substantial portion of network’s parameters. This forces the network to learn generalized representations.
In the MTL framework of Søgaard and Goldberg (2016) the underlying model is a BiLSTM with several hidden layers. Then, given different tasks, each task k ‘feeds’ from one of the hidden layers in the network. In particular, the hidden states encoded in a specific layer are fed into a multiclass classifier fk. The same work has demonstrated that this MTL protocol may be successful when there is a hierarchy between tasks and ‘lower’ tasks feed from lower layers.
AM as MTL: We use the same framework STagT for modeling AM as MTL. However, we in addition train auxiliary tasks in the network— each with a distinct label set Y ′.
LSTM-ER Miwa and Bansal (2016) present a neural end-to-end system for identifying both entities as well as relations between them. Their entity
dection system is a BLC tagger and their relation detection system is a neural net that predicts a relation for each pair of detected entities. This relation module is a TreeLSTM model that makes use of dependency tree information. In addition to decoupling entity and relation detection but jointly modeling them, pretraining on entities and scheduled sampling (Bengio et al., 2015) is applied to prevent low performance at early training stages of entity detection and relation classification. To adapt LSTM-ER for the argumentative structure encoded in the PE dataset, we model three types of entities (premise, claim, major claim) and four types of relations (for, against, support, attack).
We use the feature-based ILP model from Stab and Gurevych (2016) as a comparison system. This system solves the subtasks of AM— component segmentation, component typing, relation identification and typing—independently. Afterwards, it defines an ILP model with various constraints to enforce valid argumentation structure. As features it uses structural, lexical, syntactic and context features, cf. Stab and Gurevych (2016) and Persing and Ng (2016).
Summarizing, we distinguish our framings in terms of modularity and in terms of their constraints. Modularity: Our dependency parsing framing and LSTM-ER are more modular than STagT because they de-couple relation information from entity information. However, part of this modularity can be regained by using STagT in an MTL setting. Moreover, since entity and relation information are considerably different, such a de-coupling may be advantageous. Constraints: LSTM-ER can, in principle, model any kind of— even many-to-many—relationships between detected entities. Thus, it is not guaranteed to produce trees, as we observe in AM datasets. STagT also does not need to produce trees, but it more severely restricts search space than does LSTMER: each token/component can only relate to one (and not several) other tokens/components. The same constraint is enforced by the dependency parsing framing. All of the tagging modelings, including LSTM-ER, are local models whereas our parsing framing is a global model: it globally enforces a tree structure on the token-level.
Further remarks: (1) part of the TreeLSTM modeling inherent to LSTM-ER is ineffective for our data because this modeling exploits dependency tree structures on the sentence level,
6
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
while relationships between components are almost never on the sentence level. In our data, roughly 92% of all relationships are between components that appear in different sentences. Secondly, (2) that a model enforces a constraint does not necessarily mean that it is more suitable for a respective task. It has frequently been observed that models tend to produce output consistent with constraints in their training data in such situations (Zhang et al., 2017; Héctor and Plank, 2017); thus, they have learned the constraints.
5 Experiments
This section presents and discusses the empirical results for the AM framings outlined in §4. We relegate issues of pre-trained word embeddings, hyperparameter optimization and further practical issues to the supplementary material. Links to software used as well as some additional error analysis can also be found there.
Evaluation Metric We adopt the evaluation metric suggested in Persing and Ng (2016). This computes true positives TP, false positives FP, and false negatives FN, and from these calculates component and relation F1 scores as F1 = 2TP2TP+FP+FN . For space reasons, we refer to Persing and Ng (2016) for specifics, but to illustrate, for components, true positives are defined as the set of components in the gold standard for which there exists a predicted component with the same type that ‘matches’. Persing and Ng (2016) define a notion of what we may term ‘level α matching’: for example, at the 100% level (exact match) predicted and gold components must have exactly the same spans, whereas at the 50% level they must only share at least 50% of their tokens (approximate match). We refer to these scores as C-F1 (100%) and C-F1 (50%), respectively. For relations, an analogous F1 score is determined, which we denote by R-F1 (100%) and R-F1 (50%). We note that R-F1 scores depend on C-F1 scores because correct relations must have correct arguments. We also define a ‘global’ F1 score, which is the F1score of C-F1 and R-F1.
Most of our results are shown in Table 1.
(a) Dependency Parsing We show results for the two feature-based parsers MST (McDonald et al., 2005), Mate (Bohnet and Nivre, 2012) as well as the neural parsers by Dyer et al. (2015) (LSTM-Parser) and Kiperwasser and Goldberg
(2016) (Kiperwasser). We train and test all parsers on the paragraph level, because training them on essay level was typically too memory-exhaustive.
MST mostly labels only non-argumentative units correctly, except for recognizing individual major claims, but never finds their exact spans (e.g., “tourism can create negative impacts on” while the gold major claim is “international tourism can create negative impacts on the destination countries”). Mate is slightly better and in particular recognizes several major claims correctly. Kiperwasser performs decently on the approximate match level, but not on exact level. Upon inspection, we find that the parser often predicts ‘too large’ component spans, e.g., by including following punctuation. The best parser by far is the LSTM-Parser. It is over 100% better than Kiperwasser on exact spans and still several percentage points on approximate spans.
How does performance change when we switch to the essay level? For the LSTM-Parser, the best performance on essay level is 32.84%/47.44% CF1 (100%/50% level), and 9.11%/14.45% on RF1, but performance result varied drastically between different parametrizations. Thus, the performance drop between paragraph and essay level is in any case immense.
Since the employed features of modern featurebased parsers are rather general—such as distance between words or word identities—we had expected them to perform much better. The minimal feature set employed by Kiperwasser is apparently not sufficient for accurate AM but still a lot more powerful than the hand-crafted feature approaches. We hypothesize that the LSTMParser’s good performance, relative to the other parsers, is due to its encoding of the whole stack history—rather than just the top elements on the stack as in Kiperwasser— which makes it aware of much larger ‘contexts’. While the drop in performance from paragraph to essay level is expected, the LSTM-Parser’s deterioration is much more severe than the other models’ surveyed below. We believe that this is due to the very long sequences encountered on essay level. Dependency parsers’ global view upon the data may be too complex a modeling in such situations and/or would require much more training data: the number of possible trees to consider on n tokens may just be too huge a search space when n is large.
7
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
Paragraph level Essay level
Acc. C-F1 R-F1 F1 Acc. C-F1 R-F1 F1 100% 50% 100% 50% 100% 50% 100% 50% 100% 50% 100% 50%
MST-Parser 31.23 0 6.90 0 1.29 0 2.17 Mate 22.71 2.72 12.34 2.03 4.59 2.32 6.69 Kiperwasser 52.80 26.65 61.57 15.57 34.25 19.65 44.01 LSTM-Parser 55.68 58.86 68.20 35.63 40.87 44.38 51.11 STagBLCC 59.34 66.69 74.08 39.83 44.02 49.87 55.22 60.46 63.23 69.49 34.82 39.68 44.90 50.51
LSTM-ER 61.67 70.83 77.19 45.52 50.05 55.42 60.72 54.17 66.21 73.02 29.56 32.72 40.87 45.19
ILP 60.32 62.61 73.35 34.74 44.29 44.68 55.23
Table 1: Performance of dependency parsers, STagBLCC, LSTM-ER and ILP (from top to bottom). The ILP model operates on both levels. Best scores in each column in bold (significant at p < 0.01; Two-sided Wilcoxon signed rank test, pairing F1 scores for documents).
(b) Sequence Tagging For these experiments, we use the BLCC tagger from Ma and Hovy (2016) and refer to the resulting system as STagBLCC. Again, we observe that paragraph level is considerably easier than essay level. The major benefit is again seen for relations (about 5% points increase from essay to paragraph level). Overall, STagBLCC is ∼13% better than the best parser for C-F1 and ∼11% better for R-F1 on the paragraph level. Our explanation is that taggers are simpler local models, and thus need less training data and are less prone to overfitting. Moreover, they can much better deal with the long sequences because they are largely invariant to length: every subsequence of a sequence forms a valid input to them, while the same is not true for parsers—a subsequence of a sequence does typically not form a tree.
(c) MTL As indicated, we use the MTL tagging framework from Søgaard and Goldberg (2016) for multi-task experiments. The underlying tagging framework is weaker than that of BLCC: there is no CNN which can take subword information into account and there are no dependencies between output labels: each tagging prediction is made independently of the other predictions. We refer to this system as STagBL.
Accordingly, as Table 2 shows for the essay level (paragraph level omitted for space reasons), results are generally weaker: For exact match, C-F1 values are about ∼10% points below those of STagBLCC, while approximate match performances are much closer. Hence, the independence assumptions of the BL tagger apparently lead to more ‘local’ errors. An analogous trend holds for argument relations.
Additional Tasks: We find that when we train
STagBL with only its main task—with label set Y as in Eq. (1)—the overall result is worst. In contrast, when we include the ‘natural subtasks’ “C” (label set YC consists of the projection on the coordinates (b, t) in Y) and/or “R” (label set YR consists of the projection on the coordinates (d, s)), performance increases typically by a few percentage points. This indicates that complex sequence tagging may benefit when we train a “sublabeler” in the same neural architecture, a finding that may be particularly relevant for morphological POS tagging (Müller et al., 2013). Unlike Søgaard and Goldberg (2016), we do not find that the optimal architecture is the one in which “lower” tasks (such as C or R) feed from lower layers. In fact, in one of the best parametrizations the C task and the full task feed from the same layer in the deep BiLSTM. Moreover, we find that the C task is consistently more helpful as an auxiliary task than the R task.
C-F1 R-F1 F1 100% 50% 100% 50% 100% 50%
Y-3 49.59 65.37 26.28 37.00 34.35 47.25
Y-3:YC -1 54.71 66.84 28.44 37.35 37.40 47.92 Y-3:YR-1 51.32 66.49 26.92 37.18 35.31 47.69 Y-3:YC -3 54.58 67.66 30.22 40.30 38.90 50.51 Y-3:YR-3 53.31 66.71 26.65 35.86 35.53 46.64 Y-3:YC -1:YR-2 52.95 67.84 27.90 39.71 36.54 50.09 Y-3:YC -3:YR-3 54.55 67.60 28.30 38.26 37.26 48.86
Table 2: Performance of MTL sequence tagging approaches, essay level. Tasks separated by “:”. Layers from which tasks feed are indicated by respective numbers.
On essay level, (d) LSTM-ER performs very well on component identification (+5% C-F1 compared to STagBLCC), but rather poor on relation identification (-18% R-F1). Hence, its overall F1 on essay level is considerably below that of
8
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
STagBLCC. In contrast, LSTM-ER trained and tested on paragraph level substantially outperforms all other systems discussed, both for component as well as for relation identification.
We think that its generally excellent performance on components is due to LSTM-ER’s de-coupling of component and relation tasks. Our findings indicate that a similar result can be achieved for STagT via MTL when components and relations are included as auxiliary tasks, cf. Table 2. For example, the improvement of LSTM-ER over STagBLCC, for C-F1, roughly matches the increase for STagBL when including components and relations separately (Y-3:YC-3:YR-3) over not including them as auxiliary tasks (Y-3). Lastly, the better performance of LSTM-ER over STagBLCC for relations on paragraph level appears to be a consequence of its better performance on components. E.g., when both arguments are correctly predicted, STagBLCC has even higher chance of getting their relation correct than LSTM-ER (95.34% vs. 94.17%).
Why does LSTM-ER degrade so much on essay level for R-F1? As said, text sequences are much longer on essay level than on paragraph level— hence, there are on average many more entities on essay level. Thus, there are also many more possible relations between all entities discovered in a text—namely, there are O(2m 2 ) possible relations between m discovered components. Due to its generality, LSTM-ER considers all these relations as plausible, while STagT does not (for any of choice of T ): e.g., our coding explicitly constrains each premise to link to exactly one other component, rather than to 0, . . . ,m possible components, as LSTM-ER allows. In addition, our explicit coding of distance values d biases the learner T to reflect the distribution of distance values found in real essays—namely, that related components are typically close in terms of the number of components between them. In contrast, LSTM-ER only mildly prefers short-range dependencies over long-range dependencies, cf. Figure 4.
The (e) ILP model has access to both paragraph and essay level information and thus has always more information than all the neural systems compared to. Thus, it also knows in which paragraph in an essay it is. This is useful particularly for major claims, which always occur in first or last paragraphs in our data. Still, its performance is equal to or lower than that of LSTM-ER and STagBLCC
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
2 4 6 8 10
pr ob
.c or
re ct
|d|
LSTM-ER STagBLCC
Figure 4: Probability of correct relation identification given true distance is |d|.
when both are evaluated on paragraph level.
6 Conclusion
We present the first study on neural end-to-end AM. We experimented with different framings, such as encoding AM as a dependency parsing problem, as a sequence tagging problem with particular label set, as a multi-task sequence tagging problem, and as a problem with both sequential and tree structure information. We show that (1) neural computational AM is as good or (substantially) better than a competing feature-based ILP formulation, while eliminating the need for manual feature engineering and costly ILP constraint designing. (2) BiLSTM taggers perform very well for component detection, as demonstrated for our STagT frameworks, for T = BLCC and T = BL, as well as for LSTM-ER (BLC tagger). (3) (Naively) coupling component and relation detection is not optimal, but both tasks should be treated seperately, but modeled jointly. (4) Relation detection is more difficult: when there are few entities in a text (“short documents”), a more general framework such as that provided in LSTM-ER performs reasonably well. When there are many entities (“long documents”), a more restrained modeling is preferable. These are also our policy recommendations. Our work yields new state-ofthe-art results in end-to-end AM on the PE dataset from Stab and Gurevych (2016).
Another possible framing, not considered here, is to frame AM as an encoder-decoder problem (Bahdanau et al., 2015; Vinyals et al., 2015). This is an even more general modeling than LSTM-ER. Its suitability for the end-to-end learning task is scope for future work, but its adequacy for component typing and relation identification has been investigated in research submitted during the preparation of our current work (Potash et al., 2017).
9
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899

- Strengths:  The paper is well-written and easy to understand. The methods and results are interesting.  - Weaknesses:  The evaluation and the obtained results might be problematic (see my comments below).  - General Discussion:  This paper proposes a system for end-to-end argumentation mining using neural networks. The authors model the problem using two approaches: (1) sequence labeling (2) dependency parsing. The paper also includes the results of experimenting with a multitask learning setting for the sequence labeling approach. The paper clearly explains the motivation behind the proposed model. Existing methods are based on ILP, manual feature engineering and manual design of ILP constraints. However, the proposed model avoids such manual effort. Moreover, the model jointly learns the subtasks in argumentation mining and therefore, avoids the error back propagation problem in pipeline methods. Except a few missing details (mentioned below), the methods are explained clearly.  The experiments are substantial, the comparisons are performed properly, and the results are interesting. My main concern about this paper is the small size of the dataset and the large capacity of the used (Bi)LSTM-based recurrent neural networks (BLC and BLCC). The dataset includes only around 320 essays for training and 80 essays for testing. The size of the development set, however, is not mentioned in the paper (and also the supplementary materials). This is worrying because very few number of essays are left for training, which is a crucial problem. The total number of tags in the training data is probably only a few thousand. Compare it to the standard sequence labeling tasks, where hundreds of thousands (sometimes millions) of tags are available. For this reason, I am not sure if the model parameters are trained properly. The paper also does not analyze the overfitting problem. It would be interesting to see the training and development  loss  values during training (after each parameter update or after each epoch). The authors have also provided some information that can be seen as the evidence for overfitting: Line 622  Our explanation is that taggers are simpler local models, and thus need less training data and are less prone to overfitting .  For the same reason, I am not sure if the models are stable enough. Mean and standard deviation of multiple runs (different initializations of parameters) need to be included. Statistical significance tests would also provide more information about the stability of the models and the reliability of results. Without these tests, it is hard to say if the better results are because of the superiority of the proposed method or chance.  I understand that the neural networks used for modeling the tasks use their regularization techniques. However, since the size of the dataset is too small, the authors need to pay more attention to the regularization methods. The paper does not mention regularization at all and the supplementary material only mentions briefly about the regularization in LSTM-ER. This problem needs to be addressed properly in the paper.  Instead of the current hyper-parameter optimization method (described in supplementary materials) consider using Bayesian optimization methods.  Also move the information about pre-trained word embeddings and the error analysis from the supplementary material to the paper. The extra one page should be enough for this.  Please include some inter-annotator agreement scores. The paper describing the dataset has some relevant information. This information would provide some insight about the performance of the systems and the available room for improvement.  Please consider illustrating figure 1 with different colors to make the quality better for black and white prints.  Edit:  Thanks for answering my questions. I have increased the recommendation score to 4. Please do include the F1-score ranges in your paper and also report mean and variance of different settings. I am still concerned about the model stability. For example, the large variance of Kiperwasser setting needs to be analyzed properly. Even the F1 changes in the range [0.56, 0.61] is relatively large. Including these score ranges in your paper helps replicating your work.