Controllable Text Generation in the Instruction-Tuning Era
<|reference_start|>Controllable Text Generation in the Instruction-Tuning Era: While most research on controllable text generation has focused on steering base Language Models, the emerging instruction-tuning and prompting paradigm offers an alternate approach to controllability. We compile and release ConGenBench, a testbed of 17 different controllable generation tasks, using a subset of it to benchmark the performance of 9 different baselines and methods on Instruction-tuned Language Models. To our surprise, we find that prompting-based approaches outperform controllable text generation methods on most datasets and tasks, highlighting a need for research on controllable text generation with Instruction-tuned Language Models in specific. Prompt-based approaches match human performance on most stylistic tasks while lagging on structural tasks, foregrounding a need to study more varied constraints and more challenging stylistic tasks. To facilitate such research, we provide an algorithm that uses only a task dataset and a Large Language Model with in-context capabilities to automatically generate a constraint dataset. This method eliminates the fields dependence on pre-curated constraint datasets, hence vastly expanding the range of constraints that can be studied in the future.<|reference_end|>
@article{ashok2024controllable,
    title={Controllable Text Generation in the Instruction-Tuning Era},
    author={Dhananjay Ashok, Barnabas Poczos},
    journal={arXiv preprint arXiv:2405.01490},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.01490},
    primaryClass={cs.CL cs.AI}
}