Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition
<|reference_start|>Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition: Previous approaches to the task of implicit discourse relation recognition (IDRR) generally view it as a classification task. Even with pre-trained language models, like BERT and RoBERTa, IDRR still relies on complicated neural networks with multiple intermediate layers to proper capture the interaction between two discourse units. As a result, the outputs of these intermediate layers may have different capability in discriminating instances of different classes. To this end, we propose to adapt a supervised contrastive learning (CL) method, label- and instance-centered CL, to enhance representation learning. Moreover, we propose a novel constrained multi-layer CL approach to properly impose a constraint that the contrastive loss of higher layers should be smaller than that of lower layers. Experimental results on PDTB 2.0 and PDTB 3.0 show that our approach can significantly improve the performance on both multi-class classification and binary classification.<|reference_end|>
@article{wu2024constrained,
    title={Constrained Multi-Layer Contrastive Learning for Implicit Discourse
  Relationship Recognition},
    author={Yiheng Wu, Junhui Li, Muhua Zhu},
    journal={arXiv preprint arXiv:2409.13716},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.13716},
    primaryClass={cs.CL}
}