Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards
<|reference_start|>Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards: Recent advances in automated essay scoring (AES) have shifted towards evaluating multiple traits to provide enriched feedback. Like typical AES systems, multi-trait AES employs the quadratic weighted kappa (QWK) to measure agreement with human raters, aligning closely with the rating schema; however, its non-differentiable nature prevents its direct use in neural network training. In this paper, we propose Scoring-aware Multi-reward Reinforcement Learning (SaMRL), which integrates actual evaluation schemes into the training process by designing QWK-based rewards with a mean-squared error penalty for multi-trait AES. Existing reinforcement learning (RL) applications in AES are limited to classification models despite associated performance degradation, as RL requires probability distributions; instead, we adopt an autoregressive score generation framework to leverage token generation probabilities for robust multi-trait score predictions. Empirical analyses demonstrate that SaMRL facilitates model training, notably enhancing scoring of previously inferior prompts.<|reference_end|>
@article{do2024autoregressive,
    title={Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with
  Scoring-aware Multiple Rewards},
    author={Heejin Do, Sangwon Ryu, Gary Geunbae Lee},
    journal={arXiv preprint arXiv:2409.17472},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.17472},
    primaryClass={cs.CL cs.AI}
}