Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation
<|reference_start|>Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation: Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods. Nonetheless, a comprehensive compositional generalization evaluation benchmark of MCTG is still lacking. We propose CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a crafted three-dimensional evaluation protocol, to holistically evaluate the compositional generalization of MCTG approaches. We observe that existing MCTG works generally confront a noticeable performance drop in compositional testing. To mitigate this issue, we introduce Meta-MCTG, a training framework incorporating meta-learning, where we enable models to learn how to generalize by simulating compositional generalization scenarios in the training phase. We demonstrate the effectiveness of Meta-MCTG through achieving obvious improvement (by at most 3.64%) for compositional testing performance in 94.4% cases.<|reference_end|>
@article{zhong2024benchmarking,
    title={Benchmarking and Improving Compositional Generalization of Multi-aspect
  Controllable Text Generation},
    author={Tianqi Zhong, Zhaoyi Li, Quan Wang, Linqi Song, Ying Wei, Defu Lian,
  Zhendong Mao},
    journal={arXiv preprint arXiv:2404.04232},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.04232},
    primaryClass={cs.CL}
}